Annealed Backward–Backward Splitting for Training-free Diffusion Guidance

Training-free guidance specifies a condition for a pretrained diffusion model via a differentiable loss without additional training, and is therefore applicable across tasks and domains. Most existing methods add to each reverse step a gradient step on the loss, whose step size controls how far the guided iterate moves. Choosing this step size is difficult. The standard rule sets it from the smoothness of the loss, but this smoothness is often unavailable (e.g., when the loss involves a pretrained network), so the step size must be tuned separately for each task. In this paper, we remove the step size from the design. We formulate conditional generation as a sequence of subproblems indexed by the noise level, and we address each subproblem by composing two proximal steps, one for the prior and one for the condition, so that each subproblem (rather than step size) determines how far the guided iterate moves. In training-free guidance, we cannot compute either proximal step in closed form, because the prior is available only through the pretrained diffusion model and the loss may involve a pretrained network. We therefore approximate the proximal step for the prior by one reverse step, whose scale we identify from the noise schedule with a bound on its approximation error, and the proximal step for the condition by an inexact proximal step. We refer to the resulting method as ABBS (annealed backward-backward splitting). In our experiments, ABBS achieves the best guidance validity on three of four image tasks, in data space and latent space, with fidelity comparable to training-free guidance baselines. ABBS also applies to human motion editing, where it achieves editing performance on par with the strongest motion-editing baseline at a substantially lower guidance cost.