PSTune: Text-to-Image Reward Updates That Transfer Only When Prompt and State Match

Reward fine-tuning raises a text-to-image model’s score under a learned preference reward but often lowers what the reward does not measure: tuning a LoRA on FLUX.1-dev with HPSv2.1 reduces GenEval accuracy from 67.5% to 63.1% and within-prompt LPIPS from 54.1 to 46.6. We trace this loss to how a reward update spreads. The gradient from one image changes weights that every image uses, and its first-order effect on another image is set by the cross-Jacobian of the two generations. For an adapter that reads concatenated prompt and state features, this cross-Jacobian is the sum of a prompt similarity and a state similarity, so the update reaches any image that matches in either; it should reach an image only as far as both match. PSTune builds this rule into one three-way tensor that maps the normalized prompt embedding and the centered latent–velocity feature at each location to a velocity correction for a frozen generator. With detached features and scalar SGD, the change one image’s update causes in another factors exactly into prompt similarity, state overlap, and the source gradient, and centering removes the image-wide gradient term that would bypass this product. On FLUX.1-dev, SD3.5-Medium, and SDXL, PSTune raises GenEval ImageReward over matched LoRA by 0.29–0.31 with direct training and 0.20–0.25 with policy gradients, at the frozen model’s accuracy and LPIPS.