acceptodds
Under review as a conference paper at ICLR 2027

Elliptic Sampling: Optimal interpolation Schedules for Gaussians and Beyond

Abstract

Generative modeling based on stochastic interpolants requires specifying two key choices: an interpolant schedule bridging the initial and target distributions, and an efficient discretization of the resulting probability-flow ODE. In this work, we introduce a new framework for joint schedule/discretization design based on the analyticity properties of the (complexified) ODE trajectories, which in turn ensure good numerical behavior. Our formulation can be solved exactly via conformal mapping, yielding a natural family of variance-preserving elliptic schedules, parametrized by Jacobi elliptic functions. Pairing an elliptic schedule with a tailored discretization, we obtain Zoloflow, a new sampling algorithm closely connected to Zolotarev’s optimal rational function approximants. Crucially, both the elliptic schedule and Zoloflow explicitly incorporate the condition number of the target distribution and reduce to the classical trig schedule in the limit . For Gaussian targets, Zoloflow achieves minimax optimality among all oblivious score-span algorithms, attaining total-variation error using (exact) score queries. For the log-concave case, the elliptic schedule allows sampling with at most 2-Wasserstein-error by requiring merely score evaluations. Complementing these theoretical results, we empirically evaluate the elliptic schedule on general (non-log-concave) target distributions, including detailed comparisons with alternative schedule proposals on both synthetic benchmarks and existing image generation datasets.

Primary area:
generative models
Keywords:
Stochastic interpolants, Diffusion sampling, Interpolation schedules, Jacobi elliptic functions, Zolotarev Rational Function approximation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wRcyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes elliptic schedule, which is a stochastic interpolant schedule and corresponding discretization scheme derived for Gaussian target distributions. Theoretically, It proves minimax optimality when the target distribution is Gaussian. Furthermore they extends elliptic schedule to log-concave distributions and gives query complexity here. In experiments they show elliptic schedule can result in better performance.

Critical strengths
  • The proposed Zoloflow connects optimal interpolation schedule and rational function approximation, this is a novel and interesting idea.
  • For log-concave sampling they give theoretical analysis and query complexity, this is a solid result and the query complexity seems reliable.
Critical weaknesses
  • Elliptic schedule is derived for a class of narrow well-behaved Gaussian target distributions. For general distributions such as log-concave distributions, this paper do not give theoretical guarantee that the elliptic sampling schedule performs better than other schedules.
  • The experimental results are not comprehensive enough to demonstrate that elliptic sampling is indeed a better choice for practical generative tasks.
Questions for authors
  • I do not fully understand the Gaussian experiments. It seems that the other schedules do not converge as the number of score evaluations increases, which is a suspicious result. Can you explain it?
  • The 2D experiments seem to aim to show an improvement in NLL under the same score evaluation budget. However, the results are still poor for elliptic schedule, which raises the concern that the budget was deliberately chosen to be very small. Could you use a larger score evaluation budget and examine whether elliptic sampling can approach the ground truth while other schedules cannot?
  • The improvements in evaluation metrics on practical generative tasks appear to come more from hyperparameter search than from the elliptic schedule itself. You should provide more convincing evidence that the elliptic schedule can offer practical benefits despite the lack of a theoretical guarantee.
2 comments
AuthorsOpenReview ↗

Response to reviewer comments

Weaknesses

  1. Elliptic schedule An important clarification first. The elliptic schedule is not derived for Gaussians, it is instead motivated by the desired analyticity properties of Gaussian-smoothed diffusion scores. In the fully Gaussian case, all calculations are explicit, which directly leads to Jacobi elliptic functions. The holomorphic properties can then be transferred from the Gaussian case to more general smoothed scores; see Proposition 3.1 and Section 4.1, and the quantitative statements in Theorems 4.1 and 4.2. From the optimality properties we prove, it follows that any other schedule will necessarily have worse guarantees (provided performance in measured in this sense).
  2. More comprehensive experimental results: In our resubmission, we also add experiments using the model Stable Diffusion 3.5, which agree with the SiT model's results: Our experiments unequivocally support a quantitative advantage for the elliptic schedule, especially at small budgets, for exact and for inaccurate learned scores with errors. Altogether, we have experiments for Gaussian distributions, log-concave distributions, 2D non-log-concave toy distributions, and two different types of conditional image models with inaccurate, learned velocity fields. Our experiments support a real advantage for the elliptic schedule across budget size. We also clarify the following point:
    1. It is natural to expect the improvement of the elliptic schedule to be greater in the low step regime: As integration steps (equivalently, number of oracle calls) increases, the ODE is being solved more and more exactly. So, for a large enough number of oracle calls, all schedules achieve lower error and eventually the same performance. Therefore, the elliptic schedule matching the performance of other schedules at higher oracle budgets is fully expected (although the elliptic schedule still shows improvement in these cases). The relevant question is achieving better sample quality with fewer oracle calls than existing methods, which the experimental results support for the elliptic schedule.

Questions

  1. Yes, this is the precisely the point, not a bug! The other methods do converge, but in general are exponentially slower. Their nearly flat curves in Figure 1 reflect the much slower error decrease over the displayed range, together with a logarithmic vertical axis spanning sixteen orders of magnitude. The plot illustrates the exponential improvement of Zoloflow over all other integrators in the Gaussian regime. Another way of understanding this plot is that we are really showing how powerful optimal Zolotarev rational approximation is. The advantage is quite real!
  2. These experiments use eight score evaluations and are intended to compare sample quality at a fixed small computational budget. This is a regime in which schedule choice can have a substantial effect on discretization error, and our objective is to obtain better samples with fewer score evaluations. First we acknowledge for these original experiments we used a heuristic for choosing the hyperparameters which we now understand is suboptimal. In the updated experiments, the reader will see that we now sweep over the hyperparameters for the 2d experiments of both the elliptic schedules and other schedules and acheive uniformly better results. For the elliptic schedule we stress that we only need to sweep over 1 of the hyperparameters, so that the tuning budgets are identical. Even so, the sample quality of the elliptic schedulemore than halves the TV distance between the samples and the true distribution when compared with other schedules, supporting a strong quantitative advantage for the elliptic schedule even when tuning budgets are equal.
  3. To be clear, elliptic functions are a family of functions parameterized by our κ\kappa. Quite literally, the extra hyparameter IS the elliptic family. The two are by definition coupled. The point of our results is that this extra parameter revealed by the elliptic functions is genuinely meaningful: It is powerful enough to simultaneously (for suitable discretizations) (a) achieve minimax optimality for Gaussians, (b) match the best known log-concave rate for score-based samplers (found in Chen et al., outsatnding paper award ICML 2026), and (c) empirically perform better than the comparison schedules in the low-oracle-budget regime (a regime which we stress is quite appealing because recent diffusion literature is interested in using fewer steps when sampling). It is important to stress also that we do not add an arbitrary extra parameter to schedules here for marginal improvement. The theoretical (and practical) gaurantees are very real.
Reviewer_wRcyOpenReview ↗

Official Comment by Reviewer wRcy

Thank you for your detailed response and the substantial improvements to the revised manuscript. The additional Stable Diffusion 3.5 experimental results are quite convincing and have addressed my main concern. I will raise my score accordingly.

Reviewer_AkQSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes jointly interpolation schedules and numerical updates for probability-flow sampling. For Gaussian targets, it reduces oblivious Score-span algorithms to rational approximation and constructs Zoloflow with a matching minimax query rate. A separate analysis gives a log-concave sampling uper bound. The authors demonstrate experiments on Gaussian targets, synthetic distributions, and image generation tasks.

Critical strengths
  • The paper is well motivated and well-written with excellent illustrations and organization.
  • The Gaussian analysis provides both a lower bound and an explicit construction attaining the stated query rate. This is a stronger result than an improvement over selected solvers alone.
  • The image experiments test the schedule separately from Zoloflows’s specialized discretization. With DPM-Solver++, the tuned elliptic schedule improves FID from 4.52 to 3.89 at 8 calls compared with the rescaled trigonometric schedule.
Critical weaknesses
  • The strongest result applies to Gaussian targets with exact score queries in an oblivious score-plan class. The log-concave results uses a different collocation solver and has no matching lower bound. The image experiment uses a learned field and DPM-Solver++. Thus, the paper does not show that the main optimality result leads to an advantage for the proposed discretization in the learned, non-Gaussian setting that motivates the work.
  • The practical gains depend heavily on parameter search. In Fig.3 , the spectrum-based heuristic gives FID-10K 43.68,compared with 31.57 for the best elliptic configuration and 33.46 for the best rescaled trigonometric configuration. The heuristic therefore does not recover the reported advantage. The main image evaluation used one pretrained model, and the tuned schedules give the same reported FID at 32 calls. This provides limited evidence for a broadly reliable sampling improvement.
Questions for authors
  • Does zoloflow retain its Gaussian advantage when score queries contain controlled errors?
  • With equal tuning budgets and a separate selection set, does the elliptic schedule consistently outperform the rescaled trigonometric schedule?
  • Do parameters selected by the proposed heuristic transfer across models or evaluation budgets without further search?
2 comments
AuthorsOpenReview ↗

Response to reiviewer comments

Weaknesses

  1. There is not an argued advantage for custom elliptic finite difference discretization for the general non-Gaussian learned score.

    1. We agree. In fact, the primary contribution of this work is the elliptic schedule itself, and secondarily, its finite difference discretization. The elliptic schedule is powerful enough to simultaneously (for suitable discretizations) (a) achieve minimax optimality for Gaussians, (b) match the best known log-concave rate for score-based samplers (found in Chen et al., ICML 2026), and (c) empirically perform better than the comparison schedules in the low-oracle-budget regime (a regime which we stress is quite appealing because recent diffusion literature is interested in using fewer steps when sampling). For practical experiments, we are not claiming that one has to try all the solvers to see the benefit of the elliptic schedule. We simply used DPM-Solver++ since it is a well-accepted state of the art baseline.
  2. Additional hyperparameters and tuning cost on real experiments.

    1. Sampling improvement: As the number of integration steps (equivalently, number of oracle calls) increases, the ODE is being solved more and more exactly. So, for a large enough number of oracle calls, all schedules achieve lower error and eventually the same performance. Therefore, the elliptic schedule matching the performance of other schedules at higher oracle budgets is fully expected. The relevant question is achieving better sample quality with fewer oracle calls than existing methods, which the experimental results support for the elliptic schedule.
    2. Equal hyperparameter flexibility: In our forthcoming resubmission, we use the same hyperparameter tuning budget for all schedules whenever samples from the target distribution are available. We find empirically that target rescaling can improve performance across schedules, so we include a rescaling parameter ℓ\ell for the comparison schedules as well. For the elliptic schedule, we estimate the rescaling parameter using our heuristic ℓ^\hat{\ell} and tune LL, which determines κ=L/ℓ^\kappa=L/\hat{\ell}. While one can still run a 2d hyperparameter sweep for the elliptic schedule if they wish to achieve more optimal results, our experiments show our ℓ^\hat{\ell} heuristic for our elliptic schedule is a strong choice, enabling each schedule to tune only one hyperparmeter selected by a sweep: LL for the elliptic schedule (ℓ^\hat{\ell} fixed) and ℓ\ell for the other schedules.
    3. Scope of the heuristic: The reviewer correctly observes that the original heuristic configuration does not recover the best elliptic FID in Figure 3. Our subsequent experiments support using the heuristic to estimate the rescaling hyperparameter ℓ\ell, while LL still requires tuning (where κ=L/ℓ\kappa = L / \ell). We clarify this distinction in the resubmission.

Questions:

  1. Our current theoretical results do not establish whether Zoloflow retains its Gaussian minimax optimality under controlled score errors. The custom discretization is designed for the Gaussian, exact-score setting, and we do not claim robustness beyond those assumptions. That said, our image-generation experiments show that the elliptic schedule remains effective with imperfect learned velocity fields.

  2. In our resubmission, the elliptic and trigonometric schedules receive the same hyperparameter tuning budget, as described above. Moreover, we use separate sets for hyperparameter selection and final evaluation. Our experiments support an advantage for the elliptic schedule at low oracle budgets, with competitive performance at larger budgets.

  3. The selected parameters do not transfer across models or different datasets, but this is to be expected since different datasets have different resolutions, covariances and scales. This observation is not new: "On the Importance of Noise Scheduling for Diffusion Models" 3.3 (Chen 2023) is a famous empirical paper describing how different datasets and models require different scaling parameters (in our notation this is ℓ\ell). However, our experiments suggest that, across oracle budgets, a single sweep at a small budget may suffice. For a fixed model and solver, parameters selected at four oracle calls remain effective at larger budgets, where the elliptic schedule also becomes less sensitive to its hyperparameters. These parameters can therefore be reused across the tested budgets without further search. In summary, after a single hyperparameter sweep for elliptic, we can generate samples freely without retuning. So for every dataset, the hyperparameter search should be thought of as an O(1) operation.

Reviewer_AkQSOpenReview ↗

Official Comment by Reviewer AkQS

Thank the authors for the rebuttal.

The distinction between the elliptic schedule and Zoloflows' Gaussian-specific discretization is clearer now. The revised SiT comparison also provides stronger evidence that the gains are not solely due to a larger tuning budget.

Therefore, I will update my assessment.

Reviewer_NgoVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a principled framework for jointly designing stochastic interpolant schedules and probability-flow ODE discretizations to improve sampling efficiency. By optimizing the complex-time analyticity of ODE trajectories, it derives a variance-preserving elliptic schedule based on Jacobi elliptic functions. Combined with a tailored discretization, the resulting Zoloflow algorithm achieves minimax-optimal score-query complexity for Gaussian targets, while the elliptic schedule also provides theoretical efficiency guarantees for strongly log-concave distributions. Experiments on synthetic distributions and ImageNet demonstrate improved sampling performance, particularly under limited function-evaluation budgets.

Critical strengths
  • The paper provides a principled theoretical foundation for schedule design, rigorously proving that the proposed elliptic schedule maximizes the analyticity region of Gaussian probability-flow ODE trajectories.

  • The theoretical analysis extends beyond Gaussian distributions, establishing rigorous sampling efficiency guarantees for strongly log-concave targets using the elliptic schedule with a high-order ODE solver.

  • Gaussian experiments clearly demonstrate the effectiveness of jointly designing the schedule and discretization, outperforming competing methods.

Critical weaknesses

1. Limited applicability of theoretical guarantees to real-world Generation task

The theoretical guarantees provide limited justification for the method's effectiveness on real-world generation tasks. The minimax optimality result applies only to Gaussian targets within a restricted algorithm class, while the strongly log-concave extension provides only an upper bound and does not cover complex, multimodal image distributions. Furthermore, both results assume exact score evaluations, leaving it unclear whether these guarantees hold with imperfect neural network approximations in practice.

2. Additional hyperparameters and tuning cost

The elliptic schedule introduces an additional hyperparameter κ\kappa, with the trigonometric schedule as a special case (κ=1)(\kappa=1), raising the question of whether the observed improvements simply come from increased hyperparameter flexibility. Moreover, Figure 3 shows substantial sensitivity to (ℓ,κ)(\ell,\kappa), while the proposed heuristic performs worse than the tuned optimum. How can this sensitivity be addressed, and how expensive is the additional hyperparameter tuning, especially given that the primary goal is efficient sampling?

Questions for authors
  • Given that theoretical optimality is established only for Gaussian targets under exact score evaluations, how well do these guarantees translate to practical generation tasks? In particular, experiments on a broader range of generative models, solvers, and non-Gaussian, non-log-concave targets would help establish whether the proposed schedule offers consistent practical benefits beyond the current experimental setting.

  • Can the authors provide end-to-end computational costs, including wall-clock sampling time and hyperparameter tuning overhead? Since the elliptic schedule includes the trigonometric baseline as a special case (κ=1\kappa=1), the observed improvements may simply reflect additional hyperparameter flexibility. Can the authors demonstrate that these gains justify the additional tuning cost, ideally through comparisons with matched tuning budgets?

  • In Appendix K.1 (p. 55), the proof immediately following Example K.1 appears unrelated to the stated example.

8 comments
AuthorsOpenReview ↗

Addressing the reviewers comments

We thank the reviewer for their questions and feedback.

Weaknesses

  1. Limited applicability of theoretical guarantees to real-world generation:
    1. Algorithm class includes many widely used solvers: The algorithm class in our Gaussian minimax result is actually very broad, and covers deterministic diffusion samplers commonly used in practice. In particular, it includes the standard oblivious implementations of Euler, Heun, explicit Runge–Kutta methods (including RK4), deterministic DDIM, DEIS, and DPM-Solver(++).

    2. Significance and scope of the theoretical guarantees: We believe this assessment understates the contributions of our work over existing literature (eg. Chen et al. 2026 ICML) which only establish upper bounds on score-query complexity achieved by particular algorithms. However, whether a different schedule and sampling algorithm can improve these complexity bounds was an open question even for Gaussians. To our knowledge, our Zoloflow algorithm is the first such minimax optimal score-based sampling algorithm for Gaussians, establishing a principled link between interpolant schedules and sampling accuracy. Characterizing both the achievable complexity and a matching lower bound is therefore a substantive contribution in its own right.

      Likewise, describing the strongly log-concave extension as “only an upper bound” overlooks the significance of achieving our polylogarithmic dependence polylog(1/ε)\text{polylog}(1/\varepsilon). Recent breakthroughs by Chen et al. (2026 ICML outstanding paper award) achieve this dependence using quite different methods (first-order rejection sampling) rather than via a noise schedule modification to PF-ODE. Our result shows that the elliptic schedule enables high-accuracy sampling through a standard numerical PF-ODE solver (i.e. collocation method) and establishes rigorous and novel connections between schedule choice and score-query complexity beyond Gaussians. We emphasize that our Gaussian result indeed provides a (non-sharp) lower bound for the general log-concave case. However, a matching lower bound would be a further advance though its absence does not negate this achievement.

      We stress that, to our knowledge, the minimax score-query complexity for high-accuracy sampling of log-concave distributions, let alone complex multimodal distributions, remains unresolved. Absence of a matching lower bound in our extension therefore reflects a broader open problem in the literature, while our Gaussian result provides a complete, first of its kind, minimax characterization in its stated setting.

      Although our theoretical guarantees do not cover multimodal distributions (such as images), our approach produces an explicit, simple family of schedules whose practical value can be tested well beyond these assumptions. Our image-generation experiments demonstrate significant improvements in image quality compared to other well-known schedules at low NFE budgets with pretrained inexact scores. The contribution therefore combines provably optimal score-based sampling in the Gaussian setting, a provably high-accuracy sampler for the log-concave setting, and direct empirical evidence that the resulting schedule is practically useful for real-world image generation tasks, all within the same framework. To the best of our knowledge, no comparable results exist in the literature.

    3. Lack of theoretical guarantees for inexact score oracle: The results in the manuscript assume exact score evaluations and we do not explicitly spell out guarantees for inaccurately learned score functions. This is typical for ODE-based approaches, since vector field evaluations errors can be incorporated via standard error propagation techniques. Instead, we tackle the more relevant question of empirical testing of the elliptic schedule with pretrained models. Our image-generation experiments evaluate this setting directly and show strong performance at small oracle budgets, despite significant errors in the learned score functions.

AuthorsOpenReview ↗

continued reponse

  1. Additional hyperparameters and tuning cost on real experiments:
    1. Equal hyperparameter flexibility: In our forthcoming resubmission, we use the same hyperparameter tuning budget for all schedules whenever samples from the target distribution are available. We find empirically that target rescaling can improve performance across schedules, so we include a rescaling parameter ℓ\ell for the comparison schedules as well. For the elliptic schedule, we estimate the rescaling parameter using our heuristic ℓ^\hat{\ell} and tune LL, which determines κ=L/ℓ^\kappa=L/\hat{\ell}. While one can still run a 2d hyperparameter sweep for the elliptic schedule if they wish to achieve more optimal results, our experiments show our ℓ^\hat{\ell} heuristic for our elliptic schedule is a strong choice, enabling each schedule to tune only one hyperparmeter selected by a sweep: LL for the elliptic schedule (ℓ^\hat{\ell} fixed) and ℓ\ell for the other schedules.
    2. Hyperparameter sensitivity: Fig. 3 of the original submission shows that the choices of both ℓ\ell and κ\kappa matter for the elliptic schedule at four oracle calls. We wish to emphasize the practical implication of this sensitivity: choosing these parameters appropriately can substantially improve sample quality. The importance of the scaling choice ℓ\ell is already well known in diffusion models: Chen (2023, Sec. 3.3) showed empirically that input scaling (our ℓ\ell) can substantially affect sample quality under both linear and cosine noise schedules. Our experiments confirm the importance of scaling for the linear, trigonometric, and elliptic schedules, and further show that choosing κ\kappa appropriately can improve the elliptic schedule’s performance. However, the elliptic schedule becomes less sensitive to its hyperparameters as the number of oracle calls increases, with a broader range of configurations performing well (Fig. 15 of the original submission).
    3. *Sweep computational costs On one H200 GPU, the wall-clock time of generating samples and computing the FID score of one configuration at four oracle calls is approximately seven minutes for SiT (FID-10K) and ten minutes for SD3.5 Medium (FID-5K). Calculating FID with fewer than 10K or 5K samples could reduce this cost further. Under our heuristic, one must search over 1 hyperparamter for all schedules. We note importantly that after a single hyperparameter sweep for elliptic, we can generate samples freely without retuning. So for every dataset, the hyperparameter search should be thought of as an O(1) operation for any schedule.

Questions

  1. In our resubmission, we also added experiments using the model Stable Diffusion 3.5, which largely agree with the SiT model's results: the elliptic schedule performs consistently better at low oracle budgets and is competitive at larger budgets. Altogether, we have experiments for Gaussian distributions, log-concave distributions, 2D non-log-concave toy distributions, and two different types of conditional image models with inaccurate, learned velocity fields. Our experiments unequivocally support a quantitative advantage for the elliptic schedule at low oracle budgets, with competitive performance at larger budgets.
  2. The questions pertaining to hyperparameter tuning are addressed in the 'Additional hyperparameters and tuning cost on real experiments' section.
  3. Thanks for your note, and yes, we've removed Example K.1 in the resubmission.

Thank you for the detailed response and the additional experiments.

I acknowledge that on SiT-XL/2 the new 1D sweep (fixed ℓ̂, tuned L) shows a gain over the rescaled trigonometric schedule at a matched tuning budget, and that the issue in Appendix K has been fixed.

However, my main concern (W2) is not resolved, and the new SD 3.5M results in fact reinforce it. The response states that the SD 3.5M experiments "largely agree with the SiT model's results" and that the experiments "unequivocally support" an advantage for the elliptic schedule. Table 1 of the revision does not support this at a matched tuning budget:

  • 4 oracle calls: no matched-budget elliptic result is reported ("—"). The elliptic FID of 16.79 comes from the full 2D sweep (by my count from Fig. 22, roughly 150 configurations), compared against 1D sweeps of 17 configurations for linear (18.85) and trigonometric (26.31).
  • 8 oracle calls: the 1D elliptic sweep gives 11.56 vs. 11.76 for linear, which is essentially a tie.
  • 16 oracle calls: the 1D elliptic sweep gives 12.05, worse than both linear (10.82) and trigonometric (11.76).

In short, on the only model where the covariance heuristic is unavailable, the elliptic schedule outperforms the baselines only when given a much larger tuning budget. This is exactly the concern I raised: the gains there are not separable from the additional hyperparameter flexibility. Since the theoretical guarantees do not cover this setting, the practical claim rests on these experiments.

I therefore maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.