A Sampling-Based Optimization Method Using Gradient-Like Generators

Many applications require not a single minimizer of an energy function $f(\mathbf{x})$, but a diverse set of states in ${\mathbb{R}}^D$ whose energies follow a prescribed distribution $p^{\ast }$ - a stochastic inverse problem, solved either by sampling an explicitly constructed state-space density, or by training a neural sampler to match the induced push-forward. We instead learn a *curve*: a generator $G:[0,1]\to {\mathbb{R}}^D$ anchored at a state ${\mathbf{x}}_0$ that maps a uniform latent variable $U$ to parts of the state space with energies matching $p^{\ast }$; that is, we learn $G$ such that $f(G(U))\sim p^{\ast }$. Inverse transform sampling turns this requirement into a quantile-matching loss that upper-bounds the squared Wasserstein-2 distance between the law of $f(G(U))$ and $p^{\ast }$, with equality exactly when $f\circ G$ is non-decreasing; the curve then sweeps the level sets of $f$ in order, crossing each one exactly once between its endpoints. We call such generators *gradient-like* and they are not unique. We select the one of least kinetic energy and characterize when it exists. Freely choosing $p^{\ast }$ and the law of ${\mathbf{x}}_0$ gives two dials - which energies the curve visits, and where in state space it is anchored -and our experiments turn both. Concentrating $p^{\ast }$ near the optimum turns the sampler into a diverse optimizer: on multi-modal benchmarks $p^{\ast }$ sets how deep the curves go and the law of ${\mathbf{x}}_0$ sets how widely they spread, giving mode coverage comparable to a continuous GFlowNet with additional control over the energy distribution. In image space ($D=3072$), with the classifier's log-probability of the true class as energy, a curve anchored at a clean image interpolates smoothly from the image to a successful attack.