Learning Sampling Parameters for Diffusion Models

Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, classifier-free guidance scales, and noise schedules. These parameters are typically manually chosen once and held fixed across prompts and denoising timesteps, despite different prompts and stages of generation benefiting from different parameter values. We introduce LeSAMP a framework for learning prompt-conditioned, timestep-varying sampling parameters while keeping the diffusion model frozen. We formulate parameter selection as a reinforcement learning problem: given a user prompt, a large language model is trained to emit schedules for the chosen sampling parameters. We optimize our model using rewards from human preference models and VLM-as-a-judge. We evaluate LeSAMP on multiple diffusion backbones and find that compared to baselines, LeSAMP has a win rate of up to 68.12% using human preference scores and 73.37% using VLM-as-a-judge. These gains are validated in a user study where we achieve win rates of up to 59.46% over previous baselines. Our results suggest that learned sampling-parameter policies provide a complementary approach to existing post-training methods for improving diffusion model outputs.