acceptodds
Under review as a conference paper at ICLR 2027

Score-Based Outlier Generation without Retraining via Likelihood Reweighting

Abstract

Outlier generation plays a fundamental role in stress testing, risk analysis, and trustworthy machine learning. Existing methods typically require specialized architectures or training objectives designed specifically for this task. Instead, we show that a score-based diffusion model trained only on regular data can be turned into a controllable outlier generator without retraining or fine-tuning. We adopt recent insights that formulate outlier generation as reweighting the one-dimensional distribution of log-likelihood values induced by the pretrained model, and introduce exponential tilting as a tractable realization of this idea. This yields an explicit scalar modification of the diffusion score that steers sampling toward lower-likelihood regions. We establish a quantitative relationship between the magnitude of the exponential tilting parameter and the resulting outlier strength, and show that the controlled sampling dynamics retain convergence toward the data manifold. Across nine real-world multivariate time-series datasets, our method ranks first or second in all main benchmark comparisons against specialized outlier-generation methods, while preserving marginal, temporal, and cross-feature structure. These results demonstrate that effective outlier generation can be achieved through likelihood reweighting alone, without training a specialized generative model.

Primary area:
generative models
Keywords:
Score-Based Diffusion Models, Outlier Generation, Distribution Reweighting, Exponential Tilting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UuWjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a controllable outlier generation framework based on likelihood reweighting for pretrained score-based diffusion models. Instead of perturbing samples in the high-dimensional data space, the authors formulate outlier generation as reweighting the one-dimensional log-likelihood distribution L=(l)#μL=(l)_{\#}\mu, and construct the target distribution η\eta via exponential tilting. They show that the resulting distribution satisfies first-order stochastic dominance (FOSD) and provide a calibration between the tilting parameter λ\lambda and the Wasserstein-1 outlier strength ρ\rho. At the diffusion level, they show that likelihood reweighting modifies the pretrained score only through a scalar correction along the log-likelihood coordinate; for exponential tilting this correction is available in closed form, ct(u)=−λe−Btc_t(u)=-\lambda e^{-B_t}, yielding an inference-time controller (Eq. 12) that requires neither retraining nor an auxiliary model. They further provide a manifold convergence analysis showing that the controlled PF-ODE still converges to the data manifold for λ∈[0,1)\lambda\in[0,1). Experiments are conducted on nine real-world multivariate time-series datasets, compared against several extreme/outlier-oriented GAN baselines, reporting marginal KS, Tail-MSLE95_{95}, and additional ACF, correlation error, FGD, and CCDF area metrics, together with an ablation over λ\lambda.

Critical strengths

The empirical study covers nine real-world multivariate time-series datasets with multiple baselines, metrics, and an ablation over the control strength, which makes the evaluation reasonably broad. The method itself is simple and practical: the controller only requires a scalar rescaling of the pretrained score, with no retraining or auxiliary network. The theoretical part provides a fairly complete derivation of stochastic ordering, Wasserstein calibration, and manifold convergence, and the appendix gives detailed proofs and implementation details.

Critical weaknesses
  1. Limited novelty and insufficient comparison with related work. The method essentially introduces exponential tilting into an existing likelihood-reweighting framework, which ultimately amounts to a time-dependent score rescaling. The paper does not sufficiently clarify its substantive difference from existing temperature sampling and score-rescaling methods, nor does it provide direct comparisons.

  2. A key gap exists between the theoretical target and the actual sampling procedure. Deriving the score of the target distribution at each time step does not guarantee that plugging it into the sampling ODE will generate that target distribution. A standard Gaussian example can illustrate this inconsistency, so the stochastic ordering and Wasserstein calibration results for the target distribution cannot be directly taken as guarantees for the actual algorithm.

  3. The experiments do not sufficiently demonstrate the effectiveness of the proposed addition. Training and evaluation use the same set of real data, which limits conclusions about generalization; moreover, the uncontrolled diffusion model already achieves the main ranking results highlighted in the main text. The controller also worsens tail or temporal metrics on some datasets, so the current results do not adequately disentangle the advantage of the base diffusion model from the contribution of the added control strategy.

Questions for authors
  1. Can the authors provide a concrete counterexample or condition under which the sampling ODE with per-time score matching does not converge to the target reweighted distribution? Can the paper characterize the conditions (e.g., additional regularity, temporal consistency, or endpoint conditions) under which this construction is valid?

  2. What is the essential difference between the controller in Eq. (12) and existing temperature/score-scaling sampling schemes (e.g., multiplying sθs_\theta by a constant or time-varying coefficient)? Can the authors provide a direct empirical comparison to isolate the benefit of exponential tilting over simple score rescaling?

  3. Since training and evaluation use the same data, can the authors provide results on a held-out test set? In addition, can they report per-dataset comparisons between λ=0\lambda=0 and λ=0.12\lambda=0.12, and clearly state on which datasets and metrics the controller provides a significant gain?

3 comments

We thank the reviewer for the careful theoretical assessment and for recognizing the method’s practicality and breadth of evaluation. Your comments motivated additional diffusion baselines, direct comparisons with other score-scaling schedules, and chronological held-out evaluation. Below, we clarify the scope of our theoretical claims and explain how the expanded experiments isolate the controller’s contribution.

1. Novelty and comparisons with existing methods

At the implementation level, Eq. (12) is a time-dependent score rescaling, motivated by exponential tilting. In our updated experiments (Appendix J), we provided comparisons with the diffusion baselines suggested by Reviewer QNZs: FIDE and HTDM. Particularly, HTDM uses a Student-t-based diffusion formulation for heavy-tail modeling. Our comparison evaluates whether a scalar inference-time modification of a pretrained DM can remain competitive with such specialized diffusion methods. Our method continues to achieve competitive performance, ranking first or second on most evaluation metrics despite requiring no dedicated training.

Appendix K compares constant, linear, exponential, and reciprocal score-scaling schedules in a Gaussian-mixture setting. At the common coefficient λ=0.5\lambda=0.5, constant and linear schedules are unstable and produce excessive sample magnitudes, while exponential and reciprocal schedules remain stable. The justification of this observation is provided in Appendix K. This comparison demonstrates the importance of the temporal schedule. The stable reciprocal alternative also shows that stable behaviour is achievable with other scaling; however, our exponential scaling is distinguished by being well-motivated by the likelihood-tilting formulation.

2. Target-distribution theory and the actual sampling dynamics

Our theoretical results characterize the intended likelihood-reweighted target and motivate the score modification. They do not guarantee that the sampling dynamics recover this target ρ\rho-outlier measure exactly, and we do not claim such a guarantee. This is left for future work, as stated in the conclusion. We therefore assess whether the implemented sampler exhibits the intended likelihood steering through direct empirical evaluation.

Figure 3 shows that the Wasserstein-1 distance between the generated likelihood distributions and the λ=0\lambda=0 distribution increases monotonically with λ\lambda, which is in line with our theory. Figure 4 shows strict leftward shifts in the generated likelihood CDFs as λ\lambda increases. Together, these results provide empirical evidence that increasing λ\lambda yields samples with progressively lower likelihood under the reference model.

Tables 7–9 in Appendix G provide complementary evidence from the Tail-MSLE metric. Across the tested values of λ\lambda, Tail-MSLE decreases as λ\lambda increases on every dataset except Weather. Since Tail-MSLE measures agreement with empirical tail magnitudes, this trend suggests that the likelihood shift can also improve tail fidelity, although the effect depends on the dataset.

3. Held-out generalization and the controller’s independent contribution

Appendix J now evaluates all nine datasets using an 80/20 chronological train/test split, with every metric computed against the held-out test period. Tables 11–13 report λ=0\lambda=0 and λ=0.12\lambda=0.12 side by side, allowing the controller’s contribution to be assessed using the same pretrained model.

As detailed in the table in our response to Reviewer Cu8F, held-out Tail-MSLE95 decreases on seven of nine datasets, with reductions of up to 21.1%. The unweighted average of the dataset-wise percentage reductions is approximately 6.27%, including the deterioration on Weather and the unchanged result on Exchange at reported precision. The tables report variability alongside these descriptive improvements.

We agree that the base DM accounts for much of the strong overall benchmark performance. The controlled-versus-uncontrolled comparisons establish the additional effect of modifying its sampler, while the likelihood CDFs and W1W_1 results demonstrate adjustable outlierness. Dependence and fidelity metrics exhibit trade-offs, so we do not claim that increasing λ\lambda improves every metric. The practical result is that a single pretrained model supports increasingly low-likelihood generation and improves Tail-MSLE on several datasets, without another training run.

We hope these clarifications and expanded comparisons address the concerns about theoretical scope, novelty, and empirical attribution, and provide a basis for reconsidering your recommendation.

Thank you for your theoretical response. I have decided to maintain my score.

AuthorsOpenReview ↗

Clarification of Revisions and Scope

Thank you for the continued discussion. Our revisions are mostly focused on empirical results and numerical experiments, so we believe some miscommunication remains. We would like to provide a clarification of our scope and positioning.

To our knowledge, we are the first to show that a DM can be steered towards low-likelihood samples without specialized training or fine-tuning. The revised experiments compare against the uncontrolled model, DM baselines, and alternative score-scaling schedules, and evaluate on a chronological 80/20 split.

In the Gaussian setting with arbitrary covariance, one can show that ρ(λ)≥Var(L0)λ\rho(\lambda)\geq Var(L_0)\lambda holds for all control parameters λ∈(0,1)\lambda\in(0,1). For arbitrary distributions, however, this inequality remains open. We do not claim such convergence; rather, our experiments provide empirical support for the inequality (Figures 3 and 4).

As mentioned in our rebuttals, convergence to a ρ\rho-outlier and improving robustness (suggested by reviewer QNZs) are promising directions, but we believe they must be dealt with on a step-by-step basis. The present paper takes a first step by developing and evaluating a practical inference-time mechanism for controllable low-likelihood generation.

Reviewer_Cu8FOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores synthesizing virtual outliers using pretrained score-based diffusion models during inference. The authors first utilize the exponential tilting of the log-likelihood pushforward measure to outlier the target distributions. They then establish the stochastic ordering and Wasserstein calibration properties towards the targets and finally propose a time-dependent scalar rescaling of the pretrained score. Experiments on nine multivariate time-series datasets evaluate the proposed method.

Critical strengths
  1. The proposed method is simple and easy to implement. The authors only modify the scalar multiplication of the predicted noise for outlier synthesis, without any additional evaluation or training, making the method easy to implement.
  2. The authors establish a verifiable theoretical analysis of the proposed method. They provide interpretable target-distribution analysis. The stochastic ordering and moment identities for exponential tilting provide a clear characterization of the intended likelihood shift
  3. The evaluation results show the effectiveness of the proposed method, though it is simple.
Critical weaknesses
  1. Table 1 shows improvements of the proposed method, but it fails to attribute the improvements to the controller, as it onlu includes results with λ=0.12\lambda=0.12. As shown in Appendix (Table 4-6), when compared to GAN, λ=0\lambda=0 also ranks second among all the evaluation combinations. The improvements brought by the controller can vary between different scenarios, e.g., the Tail-MSLE or ACF error degrades in Financials/Weather.
  2. Evaluation on the same training data only measures the fitting property to seen distributions. It can't provide the generalization support of the method, undermining the real-world application of the method, as i.i.d. is not promised in practical scenarios.
Questions for authors

Please see the Weaknesses.

3 comments

We sincerely thank you for the careful reading and the constructive feedback. We are especially encouraged that the method’s simplicity, interpretability, and empirical effectiveness are recognized! Your two concerns motivated the explicit controlled-versus-uncontrolled comparisons and chronological held-out evaluation in Appendix J. We hope that our modifications and clarifications answer the rest of your questions.

1. Which improvements come from the controller?

We agree that the strong performance of λ=0\lambda=0 should be attributed to the pretrained DM. The controller’s contribution is the ability to change the generated likelihood distribution using that same model, without additional training. Its value should therefore be assessed through the λ\lambda ablation and the resulting trade-off between rarity and fidelity, rather than attributing all gains over GANs to the controller.

Tables 11–13 now place λ=0\lambda=0 and λ=0.12\lambda=0.12 side by side under held-out evaluation. For clarity, their Tail-MSLE95 results are summarized below; lower is better. In most datasets, increasing λ\lambda lowers the Tail-MSLE, which is an encouraging observation. The reduction in Tail-MSLE is as high as 21.1% (e.g., S&P500 Utilities), and the metric worsens only in the Weather dataset; overall, a mean reduction of 6.27%.

Datasetλ\lambda = 0λ\lambda = 0.12Change in mean
Utilities0.076 ± 0.0020.060 ± 0.00121.1% lower
Financials0.069 ± 0.0020.060 ± 0.00213.0% lower
Weather7.570 ± 0.0108.305 ± 0.0199.7% higher
Traffic0.188 ± 0.0030.182 ± 0.0033.2% lower
Illness0.531 ± 0.0090.468 ± 0.00811.9% lower
Exchange0.007 ± 0.0010.007 ± 0.001Unchanged
ETTm20.419 ± 0.0080.406 ± 0.0133.1% lower
ETTh20.279 ± 0.0170.242 ± 0.02013.3% lower
Electricity0.220 ± 0.0010.219 ± 0.0010.5% lower

The mean tail error decreases on seven of nine datasets, with the clearest relative reductions on Utilities, Financials, Illness, and ETTh2. We report these as descriptive improvements, not formal significance claims. In particular, the Electricity change is small, and Weather worsens.

We also acknowledge the dependence-metric trade-offs: ACF error increases on Weather, Illness, Exchange, ETTm2, ETTh2, and Electricity, while decreasing on Traffic. Correlation error improves on Weather, Traffic, ETTm2, ETTh2, and Electricity, with small increases on Illness and Exchange. Thus, λ\lambda = 0.12 is not uniformly preferable on every fidelity metric. Increasing rarity intentionally changes the sampling distribution; the useful result is that this control can improve held-out tail fidelity while retaining competitive overall structure, without another training run.

We also invite the reviewer to observe the trends in Tables 7-9 in Appendix G. With the exception of the Weather dataset, the tables show a clean trend of the Tail-MSLE metric decreasing as λ\lambda increases.

The W1W_1 plot in Figure 3 and the CDF plots in Figure 4 show a strict leftward shift of the likelihood distributions and a monotone increase in the W1W_1 distance as λ\lambda increases. We hope this gives a stronger argument that increasing λ\lambda directly controls the degree of outlierness.

2. Evaluation on training data and generalization.

The new evaluation uses an 80/20 chronological train/test split on all nine datasets, and every metric in Appendix J is computed against the held-out test period. FIDE and HTDM are included alongside the original baselines. This directly addresses whether the reported behavior extends beyond fitting the observed training distribution. Appendix J shows that our method continues to achieve competitive performance, ranking first or second on most evaluation metrics.

A chronological split is also relevant to the temporal setting you identify: the evaluation period follows the training period. We do not infer universal robustness to nonstationarity from this experiment, but it provides substantially stronger evidence than the original in-sample evaluation.

We hope these additions clarify both the controller’s independent contribution and its limitations, and strengthen your positive assessment of the work.

Dear Reviewer Cu8F,

Thank you again for your thoughtful feedback. We have posted our empirical results and the additional held-out comparisons. If you have any remaining questions or would like us to clarify any point, we would be glad to do so.

Thank the authors thorough responses. I have read the discussion and additional results. My original concerns are all addressed right now. Therefore, I decide to raise my score to 4 to support clear acceptance.

Reviewer_QNZsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an inference-time method that can generate low-likelihood samples from pretrained score-based diffusion models without retraining or fine-tuning. Its core idea is to formulate outlier generation as exponential tilting of a reference log-likelihood distribution. On nine multivariate time-series datasets, the method is compared with various general-purpose and specialized baselines and achieves favorable results in terms of marginal fidelity and tail fidelity.

In terms of limitations, first, the proposed exponential tilting method is not sufficiently compared with other distributional diffusion methods. Although the paper cites ``not suitable for such tasks'' as the reason for not comparing with diffusion models, several diffusion works for multivariate time series already exist (e.g., FIDE, NeurIPS 2024; Heavy-Tailed Diffusion Models, ICLR 2025). In addition, there are many other ways to modify the form of the log⁡p\log p distribution. The paper also lacks sufficient discussion of why exponential tilting is chosen, and it fails to sufficiently reveal the advantages of this controller over simpler alternatives.

Critical strengths
  1. The method is simple to implement and incurs low additional overhead. It adjusts the score of the pretrained model via a scalar coefficient that varies with diffusion time, without requiring an additional network or retraining for anomaly generation.
  2. The theoretical objective has a clear statistical interpretation. The paper defines anomaly generation through exponential tilting of a log-likelihood distribution and establishes the stochastic ordering properties of the target distribution as well as the relationship between the parameters and the Wasserstein displacement, giving the anomaly strength a clear mathematical meaning.
Critical weaknesses
  1. There is a lack of sufficient comparison with diffusion baselines and simple control methods. The main baselines are GAN-based, and extreme-aware time-series diffusion methods are not sufficiently covered. Moreover, controls such as post-generation amplitude scaling and other score scaling schedules are missing, making it difficult to distinguish the respective contributions of the base diffusion model and the proposed controller.
  2. The experiments mainly measure the fit to the empirical distribution from training, and the evidence for generalization and practical value is insufficient. Training and evaluation use the same set of real windows, lacking an independent test set and evaluation on unseen extreme events. Tail-matching metrics also cannot directly prove that the generated samples can improve downstream robustness or stress-testing performance.
Questions for authors
  1. What is the independent benefit of exponential tilting relative to simple alternatives? Can the authors compare vanilla diffusion, post-generation amplitude scaling, and different score scaling schedules under comparable anomaly strength, and add comparisons with related extreme-generation diffusion methods?

  2. Can the results generalize to data and extreme events not involved in training?

  3. In Eq.~(28) of Appendix~B.3, based on Eq.~(23) and the method of variation of parameters, the sign before the integral term should be positive rather than negative. Please check and correct it.

6 comments

We thank the reviewer for the constructive feedback and for recognizing the method’s simplicity, low overhead, and statistical interpretation. Your suggestions led directly to the new diffusion baselines, chronological held-out evaluation, and controlled comparisons in Appendices J–K, summarized in our official comment. Below, we explain how these additions address your questions.

1. What does exponential tilting contribute?

We agree with the reviewer that the contribution is an interpretable way to steer an existing model toward lower-likelihood samples without learning an additional model.

Exponential tilting specifies which samples should receive greater weight through their reference likelihood, and provides stochastic ordering and Wasserstein calibration for the intended target. Another advantage is that its logarithmic derivative is independent of the likelihood coordinate, giving a controller that requires neither likelihood evaluation during sampling nor an auxiliary network. We do not claim that alternative score schedules are inferior.

The new experiments separate three aspects of this contribution:

  • Comparison vs Vanilla Diffusion: Appendix J reports the same DM with λ=0\lambda=0 and λ=0.12\lambda = 0.12, and Appendix G reports the evaluation metrics under six different choices of λ\lambda. In most datasets, increasing λ\lambda lowers the Tail-MSLE, which is an encouraging observation. The reduction in Tail-MSLE is as high as 21.1% (e.g., S&P500 Utilities), and the metric worsens only in the Weather dataset; overall, a mean reduction of 6.27%. These comparisons isolate the effect of changing the sampler.
  • Comparison with amplitude scaling: Appendix K.1 compares our controller with λ=0.5\lambda = 0.5 and amplitude scaling with a=1.297a = 1.297, obtained from minimizing (E[log⁡p(X)−log⁡p(aX)]−E[log⁡p(X)−log⁡p(Xcontrol)])2(E[\log p(X)-\log p(aX)]-E[\log p(X)-\log p(X^{control})])^2 empirically. Despite comparable likelihood shifts, the resulting sample distributions differ: amplitude scaling increases exterior-tail mass while decreasing density in the intermodal valley. Our controller increases density in both low-likelihood regimes. Thus, our method can generate low-likelihood deviations that are not simply large-amplitude observations.
  • Alternative score schedules: Appendix K compares various score-scaling schedules in a two-Gaussian-mixture setting. At the common coefficient λ=0.5\lambda=0.5, constant and linear schedules are unstable and produce excessive sample magnitudes, while exponential and reciprocal schedules remain stable. The justification of this observation is provided in Appendix K. This comparison demonstrates the importance of the temporal schedule. The stable reciprocal alternative also shows that stable behaviour is achievable with other scaling; however, our exponential scaling is distinguished by being well-motivated by the likelihood-tilting formulation.

2. Does the method generalize beyond the training data?

Appendix J now evaluates all nine datasets using an 80/20 chronological train/test split, with metrics computed against held-out observations. Tables 11–13 include the uncontrolled model, our controlled sampler, and all baselines. We also added FIDE and HTDM to this benchmark - thanks for the suggestion. These comparisons directly evaluate our inexpensive inference-time modification against separately trained DM baselines. Appendix J shows that our method continues to achieve competitive performance, ranking first or second on most evaluation metrics - more details posted in response to Reviewer Cu8F. This provides out-of-sample evidence, including held-out tail fidelity, rather than evaluating only agreement with training observations.

While HTDM also achieves competitive performance compared to the GAN baselines, it requires training a DM from scratch for every choice of hyperparameter ν\nu corresponding to outlier strength, while we show that a single pretrained model supports different control strengths without retraining or fine-tuning.

We agree that downstream robustness requires a separate evaluation. This work is the first step in our broader research agenda: investigating how synthetic outliers can improve the robustness of algorithms and decision-making systems. The present paper develops and evaluates the generation mechanism, including its performance on held-out data; the next step is to integrate these samples into downstream training and decision-making procedures and measure their effect on robustness. We refer the reviewer to the fantastic works of Dentcheva's group, surveyed in [1], which provide a relevant foundation for pursuing this direction.

[1] Darinka Dentcheva and Andrzej Ruszczyński. Risk-Averse Optimization and Control: Theory and Methods. Springer, 2024.

3. Sign error in Manifold proof

Thank you for catching this. The integral should have a plus sign when written with limits from t to T.

This correction is included in the revised manuscript. The norm bound in Eq. (29) and the subsequent convergence rate are unchanged, since that bound uses the magnitude of the integral.

We hope the added results address the experimental gaps underlying your assessment, and we would appreciate your reconsideration in light of this evidence.

Dear Reviewer QNZs

Thanks again for your detailed feedback. We were wondering if there are any remaining questions from your side? We remain encouraged that the simplicity and practicality of our method is acknowledged, and we hope that our revised draft addressed the gaps in our experiments. If any points remain unclear, we would be happy to address them.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Additional Experiments

We thank all reviewers for their constructive feedback. In response to the experimental concerns, we have substantially expanded the paper with Appendices J and K, addressing generalization, additional diffusion baselines, alternative control schedules, and post-generation amplitude scaling. With these new results, we believe the reviewers' comments have significantly strengthened the paper.

1. Held-out evaluation and diffusion baselines (Appendix J).

We conduct an 80/20 chronological train/test split across all nine datasets, evaluating generated samples against held-out observations. We thank Reviewer QNZs for suggesting two diffusion-based baselines, FIDE and HTDM, which we implemented, alongside the original GAN-based comparisons. Our method remains highly competitive, ranking first or second on most reported metrics, while requiring only inference-time score modification rather than training a specialized DM. Notably, FIDE and HTDM require specialized model training for outlier generation, whereas our approach reuses a pretrained DM. Moreover, HTDM retrains a model from scratch for each value of its outlier parameter ν\nu.

2. Alternative control schedules (Appendix K).

We evaluate several score-scaling schedules on a simple mixture of two Gaussians example, adapted from the setting of [1]. We thank Reviewer UuWj for suggesting constant controls; we also extended to linear and reciprocal controls. At λ=0.5\lambda=0.5 or 0.20.2, constant and linear controls produce extremely large sample magnitudes, whereas our exponentially decaying controller and a residual controller produces stable low-likelihood samples.

From a probabilistic perspective, the controller needs to be converging to zero because, as t→∞t\to\infty, the RN derivative dμt/dνtd\mu_t / d\nu_t converges to 11, suggesting that its logarithm and its spatial derivative should vanish. Additionally, our exponential decay rate e−Bte^{-B_t} is motivated by the natural contraction rate of the OU semigroup.

From a numerical perspective, for large tt, the score function satisfies s(x,t)≈−xs(x,t)\approx -x. Under this approximation, the controlled PF-ODE reduces to x˙≈b(t)ct(⋅)x\dot x\approx b(t) c_t(\cdot) x, whose solution is xt=xTexp⁡(−∫tTb(s)cs(⋅)ds)x_t=x_T\exp(-\int_t^T b(s) c_s(\cdot) ds). Thus, a sufficient condition for xtx_t to remain bounded would be ∫0∞b(t)∥ct∥dt<∞\int_0^\infty b(t)\|c_t\|dt<\infty. This motivates a decaying controller.

Our experiments show that the reciprocal schedule is also stable, showing that exponential decay is not strictly necessary; our choice is additionally motivated by its direct connection to exponential likelihood tilting. And we choose the correction to vanish in the high-noise limit, reflecting the intended return to the Gaussian reference.

3. Post-generation amplitude scaling (Appendix K.1)

We thank Reviewer QNZs for suggesting amplitude scaling. We compare against the simple transformation X→aXX\to aX, calibrating a=1.297a=1.297 to match the mean log-likelihood reduction of our controller (λ=0.5\lambda=0.5). Although the two methods attain comparable log-likelihood distributions, their generated samples differ fundamentally. For the bimodal Gaussian mixture, low-likelihood regions occur both in the tails and between the two modes. Amplitude scaling increases tail mass but reduces density in the intermodal valley. In contrast, our controller increases density in both regions. This demonstrates that likelihood-based control can generate outliers beyond simple large-amplitude deviations, accounting for the geometry of the reference distribution.

Collectively, these experiments strengthen our empirical findings while highlighting the principal practical contribution: a simple, controllable, low-overhead mechanism for generating low-likelihood samples using an already-trained DM, without specialized architectures or retraining.

[1] Kulin Shah, Sitan Chen, and Adam Klivans. Learning mixtures of Gaussians using the DDPM objective. NeurIPS 2023.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.