acceptodds
Under review as a conference paper at ICLR 2027

StrideDiffusion: Accelerating Diffusion Models for Time-Series Generation

Abstract

StrideDiffusion accelerates time-series diffusion without retraining by using spectral activity to choose denoising times. The sampler tracks band power and changes in power and phase, uses a history correction on eligible jumps, and returns to fine steps near the clean endpoint. We test whether its gains come from grid placement rather than the update rule alone. On six datasets, we select configurations on validation data and compare them on separate data using three independently trained models per dataset. A uniform-grid replay matches every batch’s denoiser call count and the choice between a history correction and a deterministic DDIM update at each update index. The spectral grid has lower mean Context-FID, a feature-based distribution distance, on five datasets, with ETTh as the exception. The complete sampler also has lower mean Context-FID than DPM-Solver++ on five datasets and UniPC on all six at approximately matched call budgets, and is faster than both in synchronized sampling measurements. These gains do not extend to every quality metric. Our analysis separates DDIM displacement from the history residual and gives a local sufficient condition for DDIM. The implemented activity gate does not certify that condition. Experiments on additional backbones test spectral scheduling in other parameterizations. The results support spectral grid selection with a history update, while leaving the necessity of online adaptation open. Code repository: https://anonymous.4open.science/r/stridediff-ts

Primary area:
learning on time series and dynamical systems
Keywords:
Time-series Generation, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EApMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
1
Read review
Paper summary

The paper proposes StrideDiffusion, a training-free sampler for time series diffusion model. StrideDiffusion tracks band power and changes in power and phase, and uses a spectral gate to decide the stride of timestep to take. The results show that StrideDiffusion outperforms baseline samplers such as DPM-Solver++, UniPC, and DDIM on mean Context-FID metric among most of the datasets.

Critical strengths

The idea of using spectrum information of time series to design the diffusion sampler is interesting.

Critical weaknesses
  1. The paper is hard to read and understand. First of all, the paper uses lots of notation without explaining what each means and the motivation behind it. Secondly, the methodology is poorly explained. For example, section 3.2 introduces δt\delta_t and vtv_t and uses it as the condition on whether a band is active. However, it is not explained what these two statistics represent, and why we set the condition using the given threshold, and the definition and values of τE\tau_E, τP\tau_P, and τϕ\tau_\phi are not explained as well.
  2. The performance doesn’t seem consistently outperforming baselines in other metrics among most datasets. In the main performance table Table 1 only reports Context-FID as the metric. Other time series generation metrics such as discriminative score and predictive score are listed in Appendix A.1, while the performance are worse than at least one baseline for at least three of the five metrics on datasets Stocks, MuJoCo, ETTh and fMRI.
Questions for authors

Clarification requests:

  1. What is ϵ^prev\hat\epsilon_{prev}? “the estimate at the preceding visited time” means the previous timestep that updates to t? It would be better to define it rigorously.
  2. Where does the correction equation HH come from? What is the meaning behind it? Why should we define it this way?
  3. What do the statistics δt\delta_t and vtv_t mean? Why are they defined like the given equation?
  4. What are τE\tau_E, τP\tau_P, τϕ\tau_\phi? How do we determine these thresholds?
  5. Section 3.3 states “Thus the controller uses consecutive visited states“, so it uses the previous one state or all the previous states? If it is all the states, is that used through ϵ^prev\hat\epsilon_{prev}?
  6. Section 3.3 states “The implementation chooses a large stride if AtA_t is empty, a medium stride if only band 0 (DC) is active, and a small stride otherwise. We set the small stride to one.” What about large stride and medium stride? How large are they?
  7. Section 4 uses Dρ→ρ’D_{ρ→ρ’}, but in equation 2 DD is using two timesteps t→s not noise level ρρ. Are they the same DD? What does Dρ→ρ’D_{ρ→ρ’} mean?
6 comments
AuthorsOpenReview ↗

Response to Reviewer EApM (part 1 of 3)

Thank you for the detailed questions. We have revised and extended Section 3, and the answers to all seven questions are now in the paper. Several of the questions concern how the sampler works as a whole, so we start with the method in plain words.

The sampler stops at only some of the diffusion steps, which we call the visited times. At each visited time it first decides how far to jump. It takes the Fourier transform of the current noisy samples and groups the frequencies into four bands. For each band it checks whether the power or the phase has changed since the previous visited time. A band is called active when its relative power is above a threshold and it is still changing. These statistics are averaged over the batch, so all samples in a batch share one grid. If no band is active, the sampler takes a large jump. If only the lowest band is active, it takes a medium jump, and otherwise a single step. Once the time index is at or below the final window KK, it always takes single steps.

It then carries out the jump. The update is DDIM with one extra term that uses the noise estimate from the previous visited time. Single steps and the last step to the clean sample use plain DDIM.

Q1. What is ϵ^prev\hat\epsilon_{\rm prev}? Is it the estimate at the previous timestep that updates to tt?

Yes. It is the noise estimate at the time from which the sampler stepped to tt. More precisely, the sampler visits the times t0>t1>t2>…t_0 > t_1 > t_2 > \dots, starting from t0=T−1t_0 = T-1. When it is at tkt_k, the previous visited time is tk−1t_{k-1}, and ϵ^prev\hat\epsilon_{\rm prev} is the noise estimate that Eq. (1) gave at tk−1t_{k-1}. It was computed when the sampler was there, so it costs no extra network call. After a single step tk−1=tk+1t_{k-1} = t_k + 1, and after a jump of 50 steps tk−1t_{k-1} lies 50 steps back. At t0t_0 there is no previous estimate, so the sampler uses DDIM. Section 3.1 of the revision gives this definition.

Q2. Where does the correction HH come from, and what does it mean?

HH is the DDIM step plus one correction term, which comes from the second-order multistep update of the DPM-Solver family (Lu et al., 2022, 2025).

Write αt=ρt\alpha_t=\sqrt{\rho_t}, σt=1−ρt\sigma_t=\sqrt{1-\rho_t}, λt=log⁡(αt/σt)\lambda_t=\log(\alpha_t/\sigma_t) and h=λs−λth=\lambda_s-\lambda_t. The DDIM step of Eq. (2) can then be rewritten exactly as

xs=αsαtxt−σs(eh−1)ϵ^t.x_s=\frac{\alpha_s}{\alpha_t} x_t-\sigma_s (e^{h}-1) \hat\epsilon_t .

This step keeps the noise estimate constant during the whole jump, which is inaccurate for a long jump because the estimate changes along the way. The multistep update corrects for this. It extends the line through the last two estimates and replaces ϵ^t\hat\epsilon_t by

ϵ^t+h2hprev(ϵ^t−ϵ^prev),\hat\epsilon_t+\frac{h}{2h_{\rm prev}} (\hat\epsilon_t-\hat\epsilon_{\rm prev}),

where hprevh_{\rm prev} is the value of hh for the previous step. Our HH is this formula with the constant h0=0.1h_0=0.1 in place of hprevh_{\rm prev}.

We use a constant because our grids mix unit steps and long jumps. After a unit step hprevh_{\rm prev} is very small, so the weight h/(2hprev)h/(2h_{\rm prev}) becomes very large. The update then amplifies the noise in the difference of the two estimates. A constant keeps the weight bounded. The new experiments support this choice (Appendix M.4 of the revision). With the measured hprevh_{\rm prev} and no bound, C-FID gets worse on five of the six datasets. With a bound on hprevh_{\rm prev}, the results are close to those with h0=0.1h_0=0.1.

Q3. What do δt\delta_t and vtv_t mean, and why are they defined this way?

Both compare, one band at a time, the sample at the current visited time tt with the sample at the previous visited time uu.

δt[B]\delta_t[B] is the absolute change in the logarithm of the power of band BB. It tells us whether the amount of signal in this band has changed since the last visit. The logarithm makes the change relative, so a change of ten percent counts the same in a strong band and in a weak one.

vt[B]v_t[B] measures how far the phases in the band have turned since the last visit. For each frequency we take the phase difference between the two visits and average it over the channels. We then take the root mean square over the frequencies of the band, weighted by their power. The weights keep frequencies with almost no power from dominating. vt[B]v_t[B] tells us whether the oscillations in the band have moved or changed shape even when their power is the same. A sine wave that shifts in time is an example, since its power stays the same while its phase changes.

We use two statistics because a Fourier coefficient has a magnitude and a phase, and δt\delta_t follows the magnitude while vtv_t follows the phase. When neither has changed by more than its threshold since the last visit, we treat the band as settled and allow a longer jump. A band whose relative power is below τE\tau_E is left out of this test (see Q4).

AuthorsOpenReview ↗

Response to Reviewer EApM (part 2 of 3)

Q4. What are τE\tau_E, τP\tau_P and τϕ\tau_\phi, and how are they chosen?

τE\tau_E is the smallest relative power a band must have to be considered. The relative power of a band is its mean power per frequency, divided by the sum of these means over the four bands. A band below τE\tau_E is never active. τP\tau_P is the threshold on δt\delta_t, so the power of a band counts as changing when its logarithm has moved by at least τP\tau_P. τϕ\tau_\phi is the threshold on vtv_t. It decreases linearly to half its value by the end of sampling. For the three bands above the lowest one, vtv_t is multiplied by 1.5 before the comparison. A band is active when its relative power is at least τE\tau_E and its power or its phase is changing.

The values are chosen from six candidate settings that were fixed in advance (Appendix A, "Validation candidates"). Each setting gives the three thresholds, the stride sizes and the final window. The factor 1.5 and the decrease of τϕ\tau_\phi are the same in every setting. For each dataset we ran all six settings on a separate validation model. We kept the one with the lowest validation C-FID among those that use at most 96 network calls. The reported results come from other trained models. Any of the six settings can be selected for any dataset, and Sines and MuJoCo select the same one. Table R4 lists the selected values, which are also in Appendix M.2 of the revision.

Table R4. Settings used for the main results.

DatasetDiffusion steps TTLarge strideMedium strideSmall strideFinal window KKτE\tau_EτP\tau_Pτϕ\tau_\phi
Sines50050101200.100.010.04
Stocks50050201200.100.010.04
ETTh50050101120.700.020.08
MuJoCo100050101200.100.010.04
Energy10004051120.100.010.06
fMRI10002051200.500.010.06

Q5. Does the controller use the previous state or all previous states?

Only the previous one, as Section 3.3 of the revision now states. The controller compares the current sample with the sample at the previous visited time. The update uses one stored noise estimate, ϵ^prev\hat\epsilon_{\rm prev}, from that same time, and nothing older is kept. The word "visited" refers to the times at which the sampler stops, so after a jump the previous visited state is the state before the jump.

Q6. How large are the large and medium strides?

The large stride is 50 diffusion steps on Sines, Stocks, ETTh and MuJoCo, 40 on Energy and 20 on fMRI. The medium stride is 10 on Sines, ETTh and MuJoCo, 20 on Stocks, and 5 on Energy and fMRI. The small stride is always one. For scale, the diffusion model has 500 steps on Sines, Stocks and ETTh and 1000 steps on the other three datasets. These values are in Table R4, and Section 3.3 of the revision now gives them as well.

Q7. Is Dρ→ρ′D_{\rho\to\rho'} in Section 4 the same as Dt→sD_{t\to s} in Eq. (2)?

Yes, they are the same map, and the revision now says so at the start of Section 4. Dt→sD_{t\to s} in Eq. (2) is Dρ→ρ′D_{\rho\to\rho'} with ρ=ρt\rho=\rho_t and ρ′=ρs\rho'=\rho_s. Both denote one DDIM step from the noise level ρ\rho to the cleaner level ρ′\rho', using the clean prediction made at ρ\rho. Here ρ=αˉt\rho=\bar\alpha_t, which grows toward 1 as the sample gets cleaner. Section 4 uses noise levels because its results depend only on the two noise levels. The same formula then also covers the last step to the clean sample, where ρ′=1\rho'=1.

W1. Presentation and notation

The revised Section 3 opens with the plain description given at the start of this reply and with a notation table (Table 1). Each statistic and threshold is now followed by a sentence on what it measures and why it is there.

AuthorsOpenReview ↗

Response to Reviewer EApM (part 3 of 3)

W2. Results on the other metrics

The main results table of the revision (Table 2) now reports all five metrics, on nine trained models per dataset instead of three. Appendix M.1 gives every sampler on every dataset. Over the five metrics our sampler is best more often than any other sampler. Against each of the other samplers it is significantly better in more cases than it is worse.

The review counts, for each dataset, the metrics on which a sampler is not the best. By this count, every sampler in the main results table falls short on three or more metrics on Stocks, MuJoCo and fMRI. This includes the full DDIM sampler with 500 or 1000 steps. On ETTh the same holds for every sampler except matched uniform replay, which is our own update rule on a uniform grid. The count therefore does not tell the samplers apart. Only twice in the whole table is a sampler best on three or more of the five metrics of a dataset. One of the two cases is our sampler on Energy, with four metrics.

A clearer comparison tests each metric on each dataset, which gives 30 cells. Tables R2 and R3 show the result. Both tables also appear in our comment to all reviewers.

Table R2. All five metrics. Each comparison has 30 cells, one for each dataset and metric. The table counts the cells with a significant difference.

Compared withCells where ours is betterCells where ours is worseCells with no significant differenceWhere ours is worse
Full DDIM15114MuJoCo: HF error
UniPC, as in the submission16311fMRI: correlation, predictive. MuJoCo: HF error
UniPC, grid and order selected on validation14313fMRI: correlation, predictive. MuJoCo: HF error
Matched uniform replay11514ETTh: correlation, predictive. Sines: HF error. MuJoCo: HF error. fMRI: predictive
Uniform history, selected budget11613ETTh: C-FID, HF error. fMRI: correlation. Sines: HF error. MuJoCo: HF error. Stocks: predictive
DPM-Solver++, grid and order selected on validation9417fMRI: C-FID, correlation, HF error, predictive
DPM-Solver++, as in the submission8616fMRI: C-FID, correlation, HF error, predictive. Stocks: HF error. MuJoCo: HF error

Compared with the full DDIM sampler, ours is better in 15 cells and worse in one, with only 3 to 8 percent of the sampling time.

Table R3. The six samplers of the main results table, ranked by their nine-model mean in each of the 30 cells.

SamplerCells where it is bestCells among the best twoCells among the best three
Ours101420
DPM-Solver++61419
Uniform history, selected budget41215
Matched uniform replay4814
UniPC4610
Full DDIM2713

Our sampler has the best mean in 10 of the 30 cells, and the next sampler, DPM-Solver++, in 6. The cells where another sampler is better are listed in the last column of Table R2. Half of them are on one dataset, fMRI. Of the rest, the one that recurs most is the high-frequency error on MuJoCo, and it comes from selecting all settings by validation C-FID alone.

When the setting is selected by the average rank over the five validation metrics, MuJoCo gets a setting with 67 calls instead of 38. Its high-frequency error on the nine models then falls from 0.0077 to 0.0024, the level of the best other samplers (0.0025 for DPM-Solver++). In exchange, C-FID rises from 0.070 to 0.077 and the discriminative score from 0.028 to 0.031. Selection by five metrics picks the setting we already use on Sines, ETTh, Energy and fMRI, and a different setting with about twice the calls on Stocks. Appendix M.6 has the full comparison.

The discriminative score lies between 0.498 and 0.500 for every sampler on Stocks and Energy, so it cannot separate the samplers on these two datasets. Our sampler is best on all four other metrics on Energy. On Stocks it has the best C-FID and the best correlation, which is two of the four metrics that can separate the samplers.

Reviewer_pWDYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

StrideDiffusion is a training-free sampler for time-series diffusion models. It selects denoising timesteps based on spectral power and phase changes, combined with an approximate history update and a final fine-step window. The main evaluation separates validation-based configuration selection from testing across three independently trained checkpoints per dataset. A matched uniform-replay control preserves both per-batch function evaluations (NFE) and update-branch positions, providing a stronger comparison than average-NFE matching alone. The results support the benefit of spectral grid placement when paired with the proposed history update. However, they do not establish whether online adaptation is necessary, whether the method outperforms strong fixed nonuniform schedules, or whether its gains generalize across datasets, sequence lengths, and update rules.

Critical strengths
  1. Separate validation and confirmation data, independent training runs, shared initial noise, and a common feature encoder strengthen paired comparisons. Matching both NFE and update-branch positions is particularly valuable.
  2. Spectral scheduling lowers mean Context-FID on five of six datasets. All three checkpoints favor it on Sines, MuJoCo, Energy, and fMRI.
  3. The paper acknowledges negative results, metric disagreements, solver dependence, and theoretical limitations, while distinguishing generation-seed variability from training variability.
Critical weaknesses
  1. The value of online adaptation remains unclear. Appendix G shows that grids derived from independent noise trajectories recover much of the benefit, while online selection adds 9.4–24.4% overhead over replay.
  2. Strong schedule baselines are missing. Prior work, including Beta Sampling and Align Your Steps, already explores spectral timestep selection or solver-specific schedule optimization. The main confirmation experiment lacks comparisons with strong fixed nonuniform schedules. Beta-type and log-SNR schedules, evaluated under comparable validation budgets, would help isolate the controller’s contribution.
  3. Context-FID gains do not consistently imply broader quality improvements.
Questions for authors
  1. Under the main confirmation protocol, how does online adaptation compare with frozen donor grids and with uniform grids that preserve the same fine-step tail?
  2. Does the method outperform strong fixed nonuniform schedules under comparable configuration-search budgets?
  3. How sensitive are the results to h0, batch size, and batch composition?
  4. Do the gains persist across quality–latency curves?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pWDY (part 1 of 3)

Thank you for the review and for the concrete experiments you proposed. We ran them under the main protocol. Tables R1 to R3 are in our comment to all reviewers.

Tables R9 and R10 below and Table R5 (in our reply to Reviewer qABX) compare the online sampler with uniform, Beta, log-SNR and searched grids. All of these grids use our update rule, and some keep our final fine steps. Every trained model has a lower C-FID with the online sampler than with any of these grids on Sines, MuJoCo and Energy. On four datasets a spectral grid acquired once and frozen does as well as the online sampler. Over all five metrics our sampler is best on more dataset and metric pairs than any other sampler (Table R3).

W1 and Q1. Under the main confirmation protocol, how does online adaptation compare with frozen donor grids and with uniform grids that preserve the same fine-step tail?

Table R9 gives both comparisons.

Table R9. Mean C-FID over six trained models, the three original ones and three new ones. The number in brackets is how many of the six models have a lower C-FID with the online sampler. For six models we report the counts without significance stars.

Grid (all with our update rule)SinesStocksETThMuJoCoEnergyfMRI
Ours, online0.0193386.423.0980.0693516.610.5727
Frozen donor grid (same model, independent noise)0.02172 (6)86.42 (2)3.096 (2)0.07149 (6)16.61 (1)0.5737 (3)
Matched uniform replay0.02261 (6)86.76 (5)3.079 (2)0.07980 (6)17.11 (6)0.6467 (6)
Uniform prefix, same final steps0.02400 (6)86.40 (3)3.079 (2)0.07980 (6)17.18 (6)0.5234 (0)
Beta prefix, same final steps0.02419 (6)87.12 (6)2.726 (0)0.07359 (6)17.11 (6)0.5302 (0)
Log-SNR prefix, same final steps0.1121 (6)87.11 (6)11.11 (6)0.1315 (6)17.69 (6)0.5319 (0)

For the frozen grids, we acquired a donor grid on the same model from independent noise. Its C-FID is within 0.2 percent of the online sampler on Stocks, ETTh, Energy and fMRI. The online sampler is lower in all six models on Sines and MuJoCo. Nine models give the same result when the grid is acquired once on the validation model and reused on all others (Table R5 in our reply to Reviewer qABX).

The same-tail controls keep our final fine steps and change only the earlier part of the grid. On Sines, MuJoCo and Energy the online sampler is better than every prefix with the same tail in all six models. So the spectral placement of the early steps has its own effect there. A uniform prefix with the same tail does as well as ours on Stocks, and all three prefixes have a lower C-FID on fMRI. For these two datasets the advantage over matched uniform replay therefore comes from the final fine steps. On ETTh the tail is short and a Beta prefix is better.

In the new measurements the overhead is smaller than the range you quote from Appendix G. Online sampling takes 6 to 13 percent longer than replaying the same grid, which is 5 to 12 percent less time for replay (Section 5.1 of the revision). One acquisition costs about a quarter to a third of one generation run on five datasets. Counting this cost, a frozen grid becomes cheaper than online sampling from the second to the fifth generation run onward, depending on the dataset. On Stocks this takes until the tenth run, because the sample set is small (Appendix M.3).

Both comparisons are in Table 3 and Appendix M.3 of the revision.

AuthorsOpenReview ↗

Response to Reviewer pWDY (part 2 of 3)

W2 and Q2. Does the method outperform strong fixed nonuniform schedules under comparable configuration-search budgets?

On Sines, MuJoCo and Energy it does, in every trained model. All fixed grids use our update rule, and their number of calls is set on the validation model. The fixed family has six candidates, each run with three validation seeds, which is the same number of candidate evaluations as for our six spectral settings.

Table R10. Mean C-FID over six trained models. The number in brackets is how many of the six models have a lower C-FID with the online sampler.

Grid (all with our update rule)SinesStocksETThMuJoCoEnergyfMRI
Ours, online0.0193386.423.0980.0693516.610.5727
Best of six fixed grids, selected on validation0.02594 (6)86.54 (4)2.760 (0)0.08208 (6)17.27 (6)0.5720 (3)
Best Beta grid, selected on validation0.02520 (6)86.54 (4)2.760 (0)0.08208 (6)17.27 (6)0.5316 (0)
Uniform log-SNR grid0.1009 (6)86.74 (5)11.32 (6)0.2262 (6)17.59 (6)0.5343 (0)

The online sampler is lower than the selected fixed grid on Sines, MuJoCo and Energy in all six models. It is higher on ETTh in all six, and Stocks and fMRI are mixed. With nine models the differences on Sines, MuJoCo, Energy and ETTh are significant (Table R5). Of the two families you name, the best Beta grid has a higher C-FID than ours in all six models on Sines, MuJoCo and Energy. It has a lower one on ETTh and fMRI. The log-SNR grid has a higher C-FID than ours on five datasets, by a wide margin on Sines, ETTh and MuJoCo, and a lower one on fMRI.

Table R1 also includes DPM-Solver++ and UniPC with a grid and order selected on the validation model. The candidates are time-uniform, log-SNR and time-quadratic grids at orders 2 and 3, and the outcome is the same as with their default grids.

For schedules that are optimized on the model, we ran the dynamic-programming search of Table R5, which optimizes a fixed grid on the validation model. Align Your Steps belongs to the same class and is not part of these experiments.

W3. C-FID gains and broader quality

Tables R2 and R3 cover all five metrics on nine models. Our sampler has the best mean in 10 of the 30 cells, more than any other sampler. Against every other sampler it is significantly better in more cells than it is worse. The last column of Table R2 lists the cells where another sampler is better. Most of them are on fMRI, or are the high-frequency error on MuJoCo. Selecting the settings by five metrics brings this error to the level of the best other samplers. It takes more calls and gives a higher C-FID (see our reply to Reviewer EApM). The revised main table shows all five metrics.

AuthorsOpenReview ↗

Response to Reviewer pWDY (part 3 of 3)

Q3. How sensitive are the results to h0h_0, batch size, and batch composition?

For h0h_0, we replayed the same spectral grid with 0.05 and 0.2.

Table R11. Mean C-FID over six trained models on the same spectral grid. The number in brackets is how many of the six models have a lower C-FID with h0=0.1h_0=0.1.

History coefficientSinesStocksETThMuJoCoEnergyfMRI
h0=0.05h_0=0.050.02662 (6)86.34 (1)3.074 (2)0.08301 (6)16.68 (6)0.5319 (0)
h0=0.1h_0=0.1 (submitted)0.0193386.423.0980.0693516.610.5727
h0=0.2h_0=0.20.01926 (3)86.48 (5)3.182 (6)0.06583 (0)16.57 (0)0.6095 (6)

The value 0.1 is a middle choice. On every dataset its C-FID lies between those of 0.05 and 0.2. With 0.05 the C-FID is clearly worse on Sines and MuJoCo and better on fMRI. With 0.2 it is better on MuJoCo and Energy and worse on ETTh and fMRI. The measured log-SNR interval is covered in Table R6 in our reply to Reviewer qABX.

Table R12. Mean C-FID of the online sampler on the three original models, with the mean number of calls in brackets.

SettingSinesETThfMRI
Batch 10.01691 (37.5 calls)3.068 (16.7 calls)0.5689 (68.7 calls)
Batch 160.01701 (32.5 calls)3.101 (15.3 calls)0.5732 (62.6 calls)
Batch 64 (main protocol)0.01792 (29.9 calls)3.090 (13.8 calls)0.5728 (61.1 calls)
Batch 1280.01815 (29.8 calls)3.091 (13.8 calls)0.5729 (60.7 calls)
Batch 64, first regrouping0.01793 (30.0 calls)3.090 (13.9 calls)0.5731 (60.8 calls)
Batch 64, second regrouping0.01770 (30.2 calls)3.090 (13.9 calls)0.5743 (60.8 calls)

Across batch sizes C-FID changes by at most 1 percent on ETTh and fMRI. On Sines it is 6 percent lower at batch size 1, where the sampler uses about a quarter more calls. Regrouping the same noise changes C-FID by about 1 percent at most. Smaller batches use more calls, roughly 12 to 25 percent more at batch size 1, because the gate makes one decision for the whole batch. The revision states this dependence on the batch size.

Q4. Do the gains persist across quality and latency curves?

We ran both official solvers at 15 budgets from 10 to 96 calls on the three original models. Network calls account for most of the sampling time of every sampler compared here, so we report the curves against the number of calls.

Table R13. C-FID across budgets. A budget in the fourth and sixth column is the first one at which the solver has a C-FID at or below ours.

DatasetOur C-FIDBest C-FID of DPM-Solver++Budget at which DPM-Solver++ reaches our C-FIDBest C-FID of UniPCBudget at which UniPC reaches our C-FID
Sines0.01792 (30 calls)0.02167 (48 calls)not within 96 calls0.02182 (80 calls)not within 96 calls
Stocks86.77 (35 calls)86.92 (96 calls)not within 96 calls87.16 (80 calls)not within 96 calls
ETTh3.090 (14 calls)3.065 (10 calls)10 calls3.195 (80 calls)not within 96 calls
MuJoCo0.06481 (38 calls)0.07154 (96 calls)not within 96 calls0.07394 (96 calls)not within 96 calls
Energy16.71 (58 calls)16.25 (10 calls)10 calls16.75 (10 calls)not within 96 calls
fMRI0.5728 (61 calls)0.5203 (35 calls)20 calls0.6118 (96 calls)not within 96 calls

UniPC does not reach our C-FID at any budget on any dataset, and DPM-Solver++ does not reach it within 96 calls on Sines, Stocks and MuJoCo. DPM-Solver++ reaches it at 20 calls on fMRI and at 10 calls on ETTh and Energy. On these last two datasets its C-FID gets worse as the number of calls grows, and its high-frequency error on Energy at 10 calls is about five times ours. This suggests that the low C-FID at 10 calls does not reflect better samples. The adaptive-step DPM-Solver is in Table R8 in our reply to Reviewer qABX. The results for fixed grids at other budgets are in Appendix M.5 of the revision.

Reviewer_qABXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper proposes StrideDiffusion, a training-free sampler for time-series diffusion models.

    • At each reverse step, it applies an rFFT to the current state, partitions the spectrum into a few bands, and uses three statistics to decide which bands are ``active''.
    • It takes a large stride when no band is active, a medium stride when only the DC band is active, and a unit stride otherwise, with a forced fine-step window near the clean endpoint.
    • Larger jumps use an approximate history-corrected update (with a fixed history coefficient, not the standard variable-step DPM-Solver-2).
  • I regard this as an incremental contribution with practical value.

    • The coarse-to-fine spectral behavior of diffusion trajectories and spectrum-aware scheduling are established in prior work.
    • The paper’s own controls leave open whether its main claim, online spectral adaptation, is necessary.
Critical strengths
  • Slow inference is a real barrier for time-series diffusion, so training-free acceleration is useful, and code is released.
  • The experimental protocol is careful: configurations are selected on validation data, evaluated on independently trained checkpoints with shared noise and encoder, and compared with official DPM-Solver++ and UniPC and with a matched-NFE uniform replay. The paper reports exceptions and does not claim significance.
  • The paper is candid about limitations: it states that the implemented gate does not certify the condition in its own analysis.
Critical weaknesses
  • Adaptive step-size control is a long-standing idea for ODE/SDE solvers, yet the paper does not compare against existing adaptive or search-based timestep schedules
  • The paper’s own controls suggest that much of the gain comes from the spectrum-derived time grid combined with a particular update rule, and the necessity of online adaptation is left open.
  • Results rest on a small number of trained models, many differences are small relative to the variability, and the benefit depends on the dataset and on the control used.
  • The analysis is local and largely explanatory. As the paper acknowledges, the implemented gate does not certify the condition the analysis relies on, so the theory does not really justify the controller’s decisions.
  • The motivating Figure 1 is schematic and does not clearly match either the implemented rule or prior reports of spectral ordering.
Questions for authors
  • Can you compare the online controller with a fixed non-uniform grid (same update rule and NFE) on all datasets, and report paired tests across checkpoints?
  • Do the gains persist when the history coefficient is set to the measured log-SNR interval?
  • Why do the replay results differ between Section 5.2 and Appendix G?
5 comments
AuthorsOpenReview ↗

Response to Reviewer qABX (part 1 of 4)

Thank you for the careful review. We ran the three experiments that your questions ask for, and the main comparisons now use nine independently trained models per dataset. Tables R1 to R3 are in our comment to all reviewers.

The online sampler has a lower C-FID than matched uniform replay in all nine models on four datasets, which is significant after correction. It is also better than a frozen spectral grid in all nine models on Sines and MuJoCo. A spectral grid acquired once and then frozen does as well as the online sampler on Stocks, ETTh, Energy and fMRI. With the measured log-SNR interval, the gain over matched replay persists once the coefficient is bounded.

Q1. Can you compare the online controller with a fixed non-uniform grid (same update rule and NFE) on all datasets, and report paired tests across checkpoints?

We compared the online sampler with three kinds of fixed grid on all six datasets. All of them use our update rule, and their mean number of calls is within two of ours. The tests are paired across the nine trained models.

Table R5. Mean C-FID over nine trained models. The number in brackets is how many of the nine models have a lower C-FID with the online sampler. A star marks a significant difference in either direction (two-sided exact sign test over the models, Holm correction over the six datasets).

Grid (all with our update rule)SinesStocksETThMuJoCoEnergyfMRI
Ours, online0.0196286.373.1100.0697716.590.5718
Spectral grid acquired once on the validation model, frozen0.02416 (9)*86.37 (2)3.103 (2)0.07202 (9)*16.59 (2)0.5732 (6)
Best of six fixed grids, selected on validation0.02533 (9)*86.51 (6)2.742 (0)*0.08235 (9)*17.23 (9)*0.5707 (4)
Grid from dynamic-programming search0.02370 (9)*86.74 (8)*2.897 (0)*0.07958 (9)*17.17 (9)*0.5404 (0)*

The first fixed grid is the spectral grid itself. We ran the online sampler once on the validation model and saved the grid of the batch with the median number of calls. This grid was reused unchanged on all nine models. Its C-FID is within 0.3 percent of the online sampler on Stocks, ETTh, Energy and fMRI, and none of these differences is significant. On Stocks the two give the same C-FID in six of the nine models. The online sampler is lower in all nine models on Sines and MuJoCo. Even this frozen grid has a lower C-FID than matched uniform replay in all nine models on MuJoCo, Energy and fMRI. On Sines the gain over matched replay comes from the online gate.

The second fixed grid is the best of six analytic grids, namely uniform in time, uniform in log-SNR, and four Beta grids. The best one is selected on the validation model by the same rule as our own settings. The online sampler is lower on Sines, MuJoCo and Energy in all nine models. Stocks and fMRI show no significant difference, and on ETTh the fixed Beta(0.5, 0.5) grid is lower.

The third fixed grid comes from a dynamic-programming search, which we describe under W1. The online sampler is lower on Sines, Stocks, MuJoCo and Energy (in all nine models on Sines, MuJoCo and Energy), and higher on ETTh and fMRI.

These comparisons are in Table 3 and Appendix M.3 of the revision, and the revised paper names the datasets on which each result holds.

AuthorsOpenReview ↗

Response to Reviewer qABX (part 2 of 4)

Q2. Do the gains persist when the history coefficient is set to the measured log-SNR interval?

Yes, once the coefficient is bounded. The gain is measured against matched uniform replay, the control used in the paper. With the measured interval and no bound, the gains do not persist.

Table R6. Mean C-FID. The number in brackets is how many models have a lower C-FID with the spectral grid than with the grid of that row, under the same coefficient. Rows 1 to 4 use six trained models. Rows 5 to 7 use nine. Rows 2 and 4 use the prefix of the fixed grid selected on the validation model, followed by our final steps. A star marks a significant difference in either direction. Stars are given only for the nine-model rows.

History coefficientGridModelsSinesStocksETThMuJoCoEnergyfMRI
h0=0.1h_0=0.1 (submitted)spectral60.0193386.423.0980.0693516.610.5727
h0=0.1h_0=0.1 (submitted)fixed, same final steps60.02400 (6)87.12 (6)2.726 (0)0.07359 (6)17.11 (6)0.5234 (0)
measured interval, no boundspectral60.0228386.233.1660.0817716.870.5801
measured interval, no boundfixed, same final steps60.02059 (2)86.82 (5)3.214 (6)0.06986 (0)16.93 (5)0.5788 (1)
measured interval, boundedspectral90.0195186.293.1910.0665516.520.5814
measured interval, boundedmatched uniform90.02109 (9)*86.67 (7)3.201 (7)0.07147 (9)*16.90 (9)*0.6480 (9)*
measured interval, boundeduniform with its own update positions90.02153 (9)*86.61 (7)3.197 (5)0.07391 (9)*17.00 (9)*0.5702 (0)*

The first four rows change only the coefficient, while the grid, the number of calls and the positions of the history update stay the same. With h0=0.1h_0=0.1, the spectral grid has a lower C-FID than the fixed grid of row 2 in all six models on Sines, Stocks, MuJoCo and Energy. With the measured interval and no bound, this advantage is no longer clear (rows 3 and 4).

The reason is the ratio of consecutive steps. Our grids alternate unit steps and long jumps. On Sines, Stocks, MuJoCo and Energy, between a quarter and more than half of the history updates directly follow a unit step. At those updates the weight h/(2hprev)h/(2h_{\rm prev}) is very large. Over all history updates on these four datasets, the 90th percentile of this weight is about 20 to 26. With h0=0.1h_0=0.1 it is at most about 3.5, and on a uniform grid the median weight is 0.5. The update therefore multiplies the difference of two noise estimates by a large factor. For the same reason, variable-step multistep solvers are normally run with a bound on this ratio.

So we tested the measured interval with a bound. Six bounds were fixed in advance. Three are floors on hprevh_{\rm prev} (0.05, 0.10 and 0.20) and three are limits on the step ratio h/hprevh/h_{\rm prev} (1.5, 1+21+\sqrt{2} and 4). One bound was selected per dataset on the validation model.

Row 6 is the matched uniform grid, the control used in the paper. Compared with it, the spectral grid has a lower C-FID in all nine models on Sines, MuJoCo, Energy and fMRI. On Stocks and ETTh it is lower in seven of the nine. So the gain over matched replay does not depend on the constant 0.1. Row 7 is a second uniform control, which applies the history update at its own positions. The spectral grid is lower than this control in all nine models on Sines, MuJoCo and Energy, and the control is lower on fMRI.

We also compared the bounded measured interval with h0=0.1h_0=0.1 directly. With the bounded interval, C-FID is lower on MuJoCo and Energy, higher on ETTh and fMRI, and not significantly different on Sines and Stocks. The constant is one way to bound the extrapolation weight, and a bounded measured interval is another. Section 3.1 of the revision describes the coefficient in this way, and Appendix M.4 has the full tables.

AuthorsOpenReview ↗

Response to Reviewer qABX (part 3 of 4)

Q3. Why do the replay results differ between Section 5.2 and Appendix G?

The two use different protocols. Under the protocol of Section 5.2 we isolated two of the differences, the spectral setting and the batch size.

Table R7. Relative difference in mean C-FID between the online sampler and matched uniform replay, on the three original models. The number in brackets is how many of the three models have a lower C-FID with the online sampler.

DatasetSpectral settingCallsBatch 64Batch 128
SinesSetting of Section 5.23017.7% lower (3)16.5% lower (3)
SinesSetting of Appendix G3513.8% lower (3)13.6% lower (3)
StocksSetting of Section 5.2350.41% lower (2)0.41% lower (2)
StocksSetting of Appendix G290.06% lower (2)0.05% higher (2)
MuJoCoSetting of Section 5.23814.7% lower (3)13.8% lower (3)
MuJoCoSetting of Appendix G671.5% higher (0)1.5% higher (0)

The batch size has little effect, and the setting explains MuJoCo. Appendix G uses an earlier setting that takes 67 calls on MuJoCo, while Section 5.2 uses the validation-selected setting with 38 calls. With the setting of Appendix G, the MuJoCo comparison comes out as in Appendix G in all three models.

ETTh and Energy also differ between the two places, although they use the same setting in both. Their difference must therefore come from the other differences in protocol. Appendix G uses a single earlier checkpoint, a reference set drawn from the training pool, and an encoder fitted for each run. Section 5.2 of the revision describes Appendix G as an earlier study and states the role of the setting, and Appendix M.7 has the table.

W1. Adaptive and search-based schedules

We now compare with both. For the adaptive schedule, we ran the official adaptive-step DPM-Solver at six tolerances and counted every network call, including rejected steps.

Table R8. Mean C-FID on the three original models, with the mean number of calls in brackets.

DatasetOursAdaptive-step DPM-Solver, best of six tolerances
Sines0.01792 (30 calls)0.02265 (54 calls)
Stocks86.77 (35 calls)86.91 (73 calls)
ETTh3.090 (14 calls)3.129 (33 calls)
MuJoCo0.06481 (38 calls)0.07469 (60 calls)
Energy16.71 (58 calls)17.21 (89 calls)
fMRI0.5728 (61 calls)0.5261 (38 calls)

The best C-FID of the adaptive solver over all tolerances is higher than ours on Sines, Stocks, ETTh, MuJoCo and Energy, and it uses more calls. fMRI is the exception. Appendix M.5 of the revision has all tolerances.

For the search-based schedule, we implemented a dynamic-programming search in the style of GITS (Chen et al., 2024). It computes a fine teacher trajectory on the validation model and then chooses the grid with the smallest accumulated local error for the given number of calls. Table R5 shows the result with our update rule, and Q1 discusses it.

W2. Whether online adaptation is necessary

This is the comparison reported under Q1. The online sampler is better than a frozen spectral grid in all nine models on Sines and MuJoCo. On the other four datasets a frozen spectral grid does as well. The revised paper presents the spectral grid as the contribution and states on which datasets the online sampler is better. A frozen grid also saves the cost of the gate, since replaying a grid takes 5 to 12 percent less time than running the gate online. The role of the update rule is covered under Q2.

W3. Number of trained models and size of the differences

Each dataset now has nine models (Table R1). The tests compare the two samplers within each trained model, so differences in the overall C-FID level between models do not affect them. All nine models show a C-FID gain over matched replay on four datasets, which is significant after correction. The gains are 12.6 percent on Sines, 14.2 on MuJoCo, 2.8 on Energy and 11.7 on fMRI. Our C-FID is lower in eight of the nine models on Stocks (p = 0.078), and on ETTh the two do not differ significantly. On Sines, MuJoCo and Energy the result does not depend on the control. In all nine models our C-FID there is lower than that of every other sampler in Table R1. Section 5 of the revision states each result per dataset.

W4. Role of the analysis

The analysis shows which three quantities decide how far a band moves during a jump. They are the band norm of the current sample, the band norm of the clean prediction, and the history residual. This motivates measuring activity band by band. The paper uses the analysis to explain these quantities and uses experiments to assess the controller, and the introduction and Section 4 state this role.

Comments on the paper

3 comments
AuthorsOpenReview ↗

To all reviewers (part 3 of 3)

Changes in the revision

  1. Section 3 is revised and extended. It opens with a plain description of the method, has a notation table (Table 1), and explains each statistic and threshold.
  2. Figure 1 is replaced by measured trajectories.
  3. The main results table (now Table 2) reports all five metrics on nine models. The new experiments are in Section 5.2 (Table 3) and in Appendix M.
  4. The paper now states that the online sampler is better on Sines and MuJoCo, and that a frozen spectral grid does as well on the other four datasets. The results in Section 5 are stated per dataset.
  5. We added the related work on adaptive and optimized schedules that the reviewers named.
  6. The abstract, the introduction and the conclusion are updated to the nine-model results.
AuthorsOpenReview ↗

To all reviewers (part 2 of 3)

Main results

Table R1. Mean C-FID over nine trained models per dataset. Lower is better and the lowest value in each column is in bold. The number in brackets is how many of the nine models have a lower C-FID with our sampler than with the sampler of that row. A star marks a significant difference in either direction (p < 0.05 after correction).

SamplerSinesStocksETThMuJoCoEnergyfMRI
Ours0.0196286.373.1100.0697716.590.5718
Matched uniform replay0.02245 (9)*86.68 (8)3.063 (2)0.08135 (9)*17.06 (9)*0.6472 (9)*
Uniform history, selected budget (32 to 96 calls)0.02226 (9)*86.59 (8)*2.997 (0)*0.07558 (9)*17.16 (9)*0.7402 (9)*
DPM-Solver++, as in the submission0.02394 (9)*86.55 (7)3.125 (5)0.08005 (9)*16.92 (9)*0.5526 (0)*
DPM-Solver++, grid and order selected on validation0.02150 (9)*86.57 (8)3.125 (6)0.07791 (9)*16.74 (9)*0.5537 (0)*
UniPC, as in the submission0.02332 (9)*86.80 (8)*3.308 (9)*0.08209 (9)*17.12 (9)*0.6175 (9)*
UniPC, grid and order selected on validation0.02321 (9)*86.78 (9)*3.193 (9)*0.07957 (9)*17.12 (9)*0.5998 (9)*
Full DDIM (500 or 1000 calls)0.02184 (9)*86.57 (8)*3.230 (9)*0.07641 (9)*17.08 (9)*0.5921 (9)*

Matched uniform replay is a uniform grid with the same number of calls as our sampler and the same type of update at each step. Compared with it, our C-FID is lower in all nine models on Sines, MuJoCo, Energy and fMRI, by 12.6, 14.2, 2.8 and 11.7 percent. On Stocks it is lower in eight of the nine models (p = 0.078), and on ETTh the two do not differ significantly. The three original models and the two new groups of three models show the same direction on every dataset. Our C-FID is also lower than that of UniPC and of the full DDIM sampler on all six datasets. Compared with DPM-Solver++ it is lower on Sines, MuJoCo and Energy, higher on fMRI, and not significantly different on Stocks and ETTh.

Table R2. All five metrics. Each comparison has 30 cells, one for each dataset and metric. The table counts the cells with a significant difference.

Compared withCells where ours is betterCells where ours is worseCells with no significant differenceWhere ours is worse
Full DDIM15114MuJoCo: HF error
UniPC, as in the submission16311fMRI: correlation, predictive. MuJoCo: HF error
UniPC, grid and order selected on validation14313fMRI: correlation, predictive. MuJoCo: HF error
Matched uniform replay11514ETTh: correlation, predictive. Sines: HF error. MuJoCo: HF error. fMRI: predictive
Uniform history, selected budget11613ETTh: C-FID, HF error. fMRI: correlation. Sines: HF error. MuJoCo: HF error. Stocks: predictive
DPM-Solver++, grid and order selected on validation9417fMRI: C-FID, correlation, HF error, predictive
DPM-Solver++, as in the submission8616fMRI: C-FID, correlation, HF error, predictive. Stocks: HF error. MuJoCo: HF error

Table R3. The six samplers of the main results table, ranked by their nine-model mean in each of the 30 cells.

SamplerCells where it is bestCells among the best twoCells among the best three
Ours101420
DPM-Solver++61419
Uniform history, selected budget41215
Matched uniform replay4814
UniPC4610
Full DDIM2713

For every sampler we compare with, Table R2 has more cells where ours is significantly better than cells where it is worse. In Table R3 our sampler has the best mean in 10 of the 30 cells and is among the best three in 20. The next sampler, DPM-Solver++, is best in 6.

Online adaptation and frozen grids

Reviewers qABX and pWDY ask whether online adaptation is necessary. To test this, we ran the sampler once on the validation model and saved the time grid it produced. We then reused this frozen spectral grid on the nine other models (Table R5 in our reply to Reviewer qABX). Its C-FID is within 0.3 percent of the online sampler on Stocks, ETTh, Energy and fMRI, and none of these differences is significant. Even when frozen, it has a lower C-FID than matched uniform replay in all nine models on MuJoCo, Energy and fMRI. We therefore present the spectral grid as the contribution. The online sampler is better than the frozen grid in all nine models on Sines and MuJoCo, so online adaptation is the form that works best there. On the other four datasets the grid can be acquired once and reused, which saves the cost of the gate.

AuthorsOpenReview ↗

To all reviewers (part 1 of 3)

We thank the three reviewers and the area chair for their time. We are encouraged that the reviews value the evaluation protocol (qABX, pWDY) and find the idea of using spectral information to design the sampler interesting (EApM). The reviews asked for new experiments, and we have run them. This comment gives the results that concern more than one review, in Tables R1 to R3. The other tables are in the replies to each reviewer.

With nine trained models per dataset, our sampler has the lowest mean Context-FID (C-FID) on four of the six datasets (Table R1). Its C-FID is also lower than that of the full DDIM sampler on all six datasets, at 3 to 8 percent of the sampling time. Counting each dataset and metric as one cell, it is best in more of the 30 cells than any other sampler (Table R3).

New experiments

All new experiments follow the protocol of the paper. Settings are selected on a separate validation model and results are reported on other, independently trained models, with the three generation seeds averaged within each model.

  1. Six more trained models per dataset. With the three of the submission, each dataset now has nine independently trained models.
  2. Fixed grids with our update rule and approximately the same number of network calls. They are uniform in time, uniform in log-SNR, four Beta grids, and a grid found by dynamic-programming search in the style of GITS.
  3. Frozen spectral grids, including one that is acquired once on the validation model and reused on all nine models.
  4. Controls that keep our final fine steps exactly and change only the earlier part of the grid.
  5. Official DPM-Solver++ and UniPC with the grid type and order selected on the validation model. The same two solvers at 15 budgets from 10 to 96 calls. The official adaptive-step DPM-Solver at six tolerances.
  6. The history coefficient with the measured log-SNR interval, with and without a bound, and with h0h_0 set to 0.05 and 0.2.
  7. Batch sizes 1, 16, 64 and 128, and two random regroupings of the same noise.
  8. Selection of the settings by all five metrics.

All tests in our replies are two-sided exact sign tests over the trained models, with Holm correction over the six datasets for each comparison and metric. When all nine models agree, the corrected p-value is 0.023. With eight of nine, a result can be significant in one row of a table and not in another. The correction depends on the other five datasets of that row.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.