Inference-Time Alignment of Diffusion Models via Trust-Region Iterative Twisted Sequential Monte Carlo

We study inference-time alignment of diffusion models: steering a fixed pretrained model toward high-reward outputs by changing its sampling procedure. Existing Sequential Monte Carlo (SMC) methods often rely on the base model to propose samples, then use reward information to reweight and resample them. When high-reward outputs are rare, this can lead to weight degeneracy and require many particles. Twisted SMC addresses this problem by modifying proposals using look-ahead information, but learning effective twisting functions can be unstable in high-dimensional diffusion models. We propose Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC), which learns twisting functions through two steps. First, it constructs an intermediate trajectory distribution that moves toward the reward-tilted target while staying within a KL trust region around the current proposal. This step has a closed-form solution based on tempered importance weights. Second, it fits the twisting function to this distribution by weighted maximum likelihood. Theoretically, we characterize the optimal twist through a soft value function and show that it yields a zero-variance sampler. We prove that the exact trust-region update follows an escort path that reduces residual importance-weight variance, and that the population fitting objective is a forward-KL projection. Experiments on discrete diffusion text generation and text-to-image generation show improved primary alignment objectives under matched total sampled-trajectory budgets.