acceptodds
Under review as a conference paper at ICLR 2027

Failure Modes of the Adjoint Sampling: Instability and Slow Multimodal Adaptation

Abstract

The Adjoint Sampling has been presented as a theoretically grounded approach to learning diffusion samplers from unnormalized target densities. Its existing theory characterizes the desired optimal policy as a fixed point of an idealized update, but does not establish that the proposed iteration converges to, or is locally stable around, that fixed point. We rigorously analyze its convergence using Gaussian and Gaussian-mixture targets. Their amenability to closed-form and asymptotic analysis allows us to isolate two fundamental challenges for the iterative sampler: local stability and probability redistribution across separated modes. In the Gaussian case, we prove that, over a nontrivial parameter regime, an arbitrarily small terminal-mean error alternates in sign and grows geometrically across iterations. For well-separated Gaussian mixtures, asymptotic analysis reveals two additional failure modes with exponentially slow convergence: discovery of an unrepresented mode and correction of inaccurate mode weights. These results hold with exact conditional expectations and density evolution, separating intrinsic properties of the iteration from errors due to finite sampling or neural-network approximation. Direct quadrature and Fokker–Planck solutions reproduce the predicted failure modes. Neural-network experiments show that direct velocity damping strongly suppresses the local Gaussian instability, while replay provides weaker moderation; neither intervention reliably resolves missing-mode discovery or probability redistribution across separated modes. Additional numerical experiments revealed that our identified failure modes persist beyond the idealized Gaussian setup and are also observed in higher dimensional state spaces. Together, these results show that fixed-point correctness does not imply convergence of AS method and that stabilizing local update dynamics does not by itself resolve global probability-allocation failures. Finally, we discuss the broader implications of our findings for the design and analysis of future diffusion-based sampling algorithms.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
sampling, adjoint matching, diffusion models, stability analysis, asymptotic analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1hFkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes the convergence of Adjoint Sampling (AS), focusing on stability, exploration of a missing mode, and correction of the relative weights of discovered modes. For Gaussian and 2-mixture Gaussian models, they show that AS can be unstable for Gaussian and exponential time is needed to sample a missing mode or adjust the relative weights for 2-mixture Gaussians. Numerical experiments extend the theory to more general target distributions.

Critical strengths

The authors tackle an interesting problem. I think the paper is well organized: it introduces the reader to the problem, focuses on the theory, and then presents experiments that both illustrate the theoretical results and extend beyond that. A nice choice by the authors is to begin with the Gaussian analysis, which is relatively easy to follow. The results themselves seem significant.

Critical weaknesses

I believe the introduction, especially the paragraph about AS, is too dense. As I wasn't familiar with the approach, I found that paragraph quite hard to read. I believe the paper would benefit from using the additional page provided by the camera-ready version to make parts of the text, especially Section 2, plainer and less compressed. Section 3 is also difficult to follow, as the theoretical results are listed with explanatory comments postponed after their statements.

There is also a typo in Eq. 2: the authors should have used gg instead of ∇g\nabla g.

Questions for authors
  • How important are the equal component variances and the choice of means ±ν\pm \nu in the multimodal analysis? Would similar estimates hold for more general 2-component Gaussian mixtures?

  • Since higher dimensions and a large number of modes make the problem much harder, do you think the same theoretical analysis can be extended to such cases?

Reviewer_nbsdOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper identifies conditions under which adjoint sampling becomes unstable, deriving closed-form Gaussian update that show how terminal mean errors can be amplified across iterations. It also analyzes the missing-mode problem, showing that the probability of discovering the missing mode can be exponentially small. The experiments validate these using numerical solution of Fokker-Planck to reproduce failure modes and by conducting ablations with neural network approximations.

Critical strengths

This paper provides clear analysis and detailed derivations for each proposition. It also gives concrete case when the adjoint sampling could fail, e.g., missing mode problem.

Critical weaknesses
  1. At the beginning of section 3, the paper sets σ(t)=t\sigma(t) = t and states that this is without loss of generality, but it does not provide more justification how this would affect the result if it is time dependent. The instability assumption and threshold could also depend on σ(t)\sigma(t). It is better to provide a proof to show the transformation from a general σ(t)\sigma(t) to a constant diffusion scaling. Simply saying this is generalizable is not sufficient.
  2. The instability from theorem 3.3 is conditional but not generic. The conclusion relies on the condition when Am<−1A_m < -1, but does not show the existence of this case.
  3. This paper identifies missing mode as a limitation of adjoint sampling. From my perspective, this appears to be a generic exploration problem. When the current distribution only has one mode while the target distribution expects to be two-mode, the probability of discovering the missing mode can be exponentially small. The analysis shows that adjoint sampling does not avoid this problem, but it does not show that this failure is specific to adjoint sampling.
Questions for authors

Given the weaknesses discussed above:

The experiments in section 4 are done specific to the adjoint sampling operator, but it still does not show identifying missing mode is uniquely hard for adjoint sampling. I think it is important to compare against with some very basic baselines, at least MCMC, Langevin sampling or other diffusion samplers. It would be better to show the failure mode under the same settings in the original adjoint sampling experiments and see how frequent it would fall into the unstable parameter situations.

Reviewer_CWQvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates adjoint sampling, a popular method which learns diffusion samplers on un-normalized target densities. They emphasize on adjoint sampling by specifically focusing on two problems i) The ability to identify an under-represented mode in the training data distribution as well as ii) reweighting of imbalance in the mode weights.

Critical strengths

The authors show the slow multi-modal adaption and mode discovery as a closed form expression independent of the choice of sampler employed. The experiments are verified on Gaussian distributions of 1D. The authors conduct neural network experiments to further support their claims. Damping of the update and employment of replay are presented as plausible solutions to the problem.

Critical weaknesses

My primary concern with the manuscript in its current form is that claims made in the manuscript abstract are not sufficiently demonstrated using experimental results. This leads me to recommend rejection. Below, I provide an explanation for my stance on the manuscript.

  1. Limited Scope The main proposition Proposition 3.1 by the manuscript is constrained to 1D Gaussian mixtures. This casts doubt on the main claim which states that such instability is manifest in "well-separated gaussian mixtures". The 128D gaussian which the authors experiment on also does not demonstrate this instability.

  2. Unverified Theorems: The bounds for mode discovery and missing mode correction have been formalized in Theorem 3.5 and Theorem 3.8 however they have not been verified in the main paper.

  3. Practical Relevance: The authors highlight a limitation of adjoint sampling but do not explore the consequences on real world datasets. Havens.et.al, 2025 tests the method on molecular generation tasks which is missing from the paper. More concretely, the authors do not explain the source of such instability and why its relevant to empirical practitioners using adjoint sampling.

Questions for authors

Clarifications:

  1. In Figure 1 and Proposition 3.1 the stability is demonstrated for a 1D gaussian mixture. How would the stability criterion change if we use a higher dimensional gaussian?

  2. In Theorem 3.8, the authors argue for an exponentially weak correction of inaccurate weights. Can the authors concretely highlight how this exponential slowness affects the time for convergence in the Gaussian1D case?

  3. Can the authors explain why 128D gaussians do not demonstrate the instability predicted in Gaussian1D scenarios?

Future Work:

  1. How does Adjoint sampling fail on Molecular Generation tasks, and how does it relate to the propositions introduced in the paper?

I'm willing to raise my score if the authors are able to address my concerns regarding the scope and applicability of their approach.

Comments on the paper

1 comment
Area_Chair_ZSFJOpenReview ↗

Author-Reviewer Discussion ends in 1 day, 12 hours

Dear authors and reviewers,

Thank you to the authors for their detailed responses and the revised manuscript. The discussion period closes in about a day and a half, so this is the last chance for a direct exchange before the reviewer-AC discussion starts on Nov 19.

Reviewer nbsd: I haven’t yet seen a reply from you. Please read the authors’ response and the revision, which address the points raised in your review, and let us know whether your concerns have been addressed. Please update your recommendation if your assessment has changed. Even a short acknowledgment helps.

Reviewer CWQv: thank you for engaging. The authors replied on Nov 16 to your latest comment and added new experiments. Please let us know whether these settle your remaining points, and if some do not, which ones, so that they can be discussed before the period ends.

Reviewer 1hFk: thank you for your reply. No further action is needed unless you wish to comment on the discussion.

Authors: please be ready to answer any follow-up questions in the time left, focusing on the points most likely to affect the decision. I encourage everyone to keep the exchange focused on the technical content of the paper.

Thank you all for keeping the discussion constructive and timely.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.