acceptodds
Under review as a conference paper at ICLR 2027

SaTPO: Text-to-Image Safety Alignment via Transition Preference Optimization

Abstract

Safety alignment of text-to-image diffusion models aims to suppress harmful content without compromising useful generation capabilities. However, preferences over completed images compress multidimensional generation factors into a coarse signal, potentially driving changes beyond those required for safety. To address this, we propose Safety Transition Preference Optimization (SaTPO), which enforces step-wise alignment during denoising. SaTPO leverages the insight that an unsafe prompt and its safe rewrite naturally form a tightly coupled preference pair when conditioned on a shared parent state and noise realization. By anchoring optimization on identical parent states, SaTPO eliminates historical trajectory divergence, confining safety adaptation to an incremental alignment while preserving task-relevant semantics. Multi-scale drift constraints and benign replay further limit behavioral deviation and preserve ordinary generation. We additionally construct CoProPV, a refined dataset ensuring visually verified contrasts. Experiments on Stable Diffusion v1.5 and SDXL demonstrate improved safety-quality trade-offs. Compared with AlignGuard, PerspectiveVision unsafe-detection rates on Inappropriate Image Prompts decrease from 12.70% to 6.00% and from 15.04% to 9.30%, respectively, while FID improves from 32.28 to 25.95 and from 36.50 to 27.51.

Primary area:
generative models
Keywords:
Text-to-image diffusion models, Safety alignment, Direct preference optimization, Responsible generative Al

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_8VzNOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper investigates safety alignment for text-to-image diffusion models, focusing on the limitations of image-level preference optimization. The authors propose SaTPO, which constructs step-level safety preferences using shared denoising states and noise, together with multi-scale drift regularization and benign replay. Experiments on SD v1.5 and SDXL demonstrate improved safety performance while largely preserving benign generation quality.

Critical strengths
  1. The proposed method is well-motivated and technically sound. By constructing preference pairs from shared denoising states and noise, SaTPO reduces trajectory-induced variations and provides more controlled supervision for safety alignment.
  2. The experimental results are convincing. SaTPO demonstrates better safety performance across two diffusion backbones, multiple evaluation datasets, and safety metrics. The improvements also extend to adversarial prompts, supporting the effectiveness of the proposed approach.
  3. This paper is well-written and easy to follow.
Critical weaknesses
  1. The safety relevance of the constructed transition preferences could be better demonstrated. While shared-state comparison eliminates trajectory-induced variations, it remains unclear whether the resulting differences primarily reflect safety-related changes rather than unrelated semantic variations. Visualizing the preferred (safe) and rejected (unsafe) branches at different denoising steps could provide more direct insights.
  2. The contributions of individual drift regularization components remain unclear. The proposed method combines stepwise and trajectory-level drift penalties, but the ablation studies do not independently evaluate their effects. This makes it difficult to determine their respective contributions to the reported perfromance.
  3. The experiments are limited to SD v1.5 and SDXL, both based on U-Net architectures. Evaluating SaTPO on DiT-based models such as SD v3 would help demonstrate its applicability to more recent text-to-image generation architectures.
Questions for authors

Could the authors clarify why the unsafe-conditioned trajectory is chosen as the shared-state anchor, rather than the safe-conditioned trajectory?

7 comments
AuthorsOpenReview ↗

Official Comment by Authors(W1)

We sincerely thank you for the positive feedback and thoughtful suggestions. Below, we address the raised concerns, with details in the revised version.


W1 — Safety relevance of intermediate branches

“While shared-state comparison eliminates trajectory-induced variations, it remains unclear whether the resulting differences primarily reflect safety-related changes rather than unrelated semantic variations. Visualizing the preferred (safe) and rejected (unsafe) branches at different denoising steps could provide more direct insights.”

Following your suggestion, we have visualized both candidate branches at multiple denoising steps and measured their safety with two common readouts. We use continuation to the final image with zero conditioning (∅) and LCM lookahead with zero conditioning. Latent Consistency Models (LCM) [1] distill denoising into a few steps. We use an LCM-LoRA [2] accelerated model to preview intermediate branches in one step, following LCM-Lookahead [3], making local safety and semantic differences visible in image space. Each comparison starts from candidates with the same parent and noise.

Furthermore, Figure 2(b) provides an insight of safety preference at intermediate steps. The mean SafetyLRM gap, defined as the safe rewrite branch score minus the unsafe prompt branch score, is positive early in denoising, when it is largest, then approaches zero toward the end. This indicates that the intermediate branches contain a safety preference signal consistent with their prompt roles.

\hline \textbf{Common readout} & \begin{array}{c} \textbf{Safe branch} \\\\ \textbf{PV (\\%, ↓)} \end{array} & \begin{array}{c} \textbf{Unsafe branch} \\\\ \textbf{PV (\\%, ↓)} \end{array} & \textbf{PV gap (pp)} & \begin{array}{c} \textbf{Mean Q16} \\\\ \textbf{probability gap (pp)} \end{array} \\\\ \hline \text{∅ continuation} & \mathbf{49.38} & 57.86 & 8.48 & 7.66 \\\\ \text{LCM readout} & \mathbf{47.14} & 52.68 & 5.54 & 5.37 \\\\ \hline \end{array}$$ Each readout covers 140 prompt pairs across seven risk categories and eight timesteps, yielding 1,120 paired branches from frozen SDXL Base. Gaps are the unsafe branch minus the safe branch. Both readouts give the safe branch lower average unsafe scores. We display six key timesteps, t = 951, 851, 751, 651, 351, and 101, with denser coverage of early denoising, where the visible branch differences are more pronounced, and later views showing how they evolve. This selection is consistent with SaTPO’s greater weighting of earlier steps, where Figure 2(b) shows stronger safety separation. Figure 10 (p. 29) shows portraits that retain the subject while reducing graphic facial details, and street scenes that retain the crowd and plaza while changing harmful foreground content. Qualitative examples are shown in [Figure 10](https://anonymous.4open.science/r/satpo-rebuttal-figures-92B8/images/figure10_intermediate_branches.png). The TIFA \[4] evaluation in Appendix C.13 provides complementary evidence after training, using visual question answering to check whether generated images retain benign attributes specified in the prompts. We have revised the manuscript to include the readout protocol and results at individual timesteps in Appendix C.14 and Table 21 (pp. 26–27). **References** * \[1] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li and Hang Zhao, Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference, arXiv preprint arXiv:2310.04378, 2023 * \[2] Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolinário Passos, Longbo Huang, Jian Li and Hang Zhao, LCM-LoRA: A Universal Stable-Diffusion Acceleration Module, Technical report, arXiv:2311.05556, 2023 * \[3] Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H Bermano, Gal Chechik and Daniel Cohen-Or, LCM-Lookahead for Encoder-based Text-to-Image Personalization, ECCV 2024 * \[4] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna and Noah A Smith, TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering, ICCV 2023
AuthorsOpenReview ↗

Official Comment by Authors(W2 - Part1/2)

W2 — Step-wise and common drift

“The proposed method combines stepwise and trajectory-level drift penalties, but the ablation studies do not independently evaluate their effects.”

We have added separate and joint removals of the two drift terms, retaining benign replay and fixing the data and update budget across the three controls. Following reviewer itda’s suggestion, we also assess benign attribute retention with TIFA [1], which asks visual questions about the benign content explicitly requested in unsafe prompts. MUSIQ [2] complements this evaluation by assessing perceptual image quality without a reference image, including sensitivity to blur and other visual distortions.

Local drift constrains each candidate’s transition log ratio, while common drift constrains their accumulated signed shift across steps. The table therefore examines both individual transitions and their accumulated change, with a separate replay removal for context. Unsafe rates and TIFA are reported as percentages.

\begin{array}{lllllllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SDXL configuration}}}}\hspace{10.04262em}\vphantom{\textbf{PV}} & \rlap{\hspace{8.62492em}\textbf{I2P-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{21.22644em}{0.04em}}}\hspace{2.47767em}\vphantom{\textbf{PV}} & \hspace{2.83754em} & \hspace{2.27483em} & \hspace{5.39917em} & \hspace{4.73723em} & \rlap{\hspace{0.67994em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{6.93081em}{0.04em}}}\hspace{3.5671em}\vphantom{\textbf{PV}} & \hspace{2.86371em} & \rlap{\hspace{0em}\textbf{I2P-1000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{3.97659em}{0.04em}}}\hspace{4.47659em}\vphantom{\textbf{PV}} \\\\ \hspace{10.04262em} & \ORcell{0}{0}{\textbf{PV ↓}} & \ORcell{0}{0}{\textbf{Q16 ↓}} & \ORcell{0.11505}{0.11505}{\textbf{IP ↓}} & \ORcell{0}{0}{\textbf{NudeNet ↓}} & \ORcell{0}{0}{\textbf{MUSIQ ↑}} & \ORcell{0}{0}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} & \ORcell{0.41331}{0.41331}{\textbf{TIFA ↑}} \\\\ \hline \ORcell{0}{6.80813}{\text{SaTPO}} & \ORcell{0.21756}{0.21756}{\mathbf{9.30}} & \ORcell{0.3975}{0.3975}{\mathbf{4.10}} & \ORcell{0.11614}{0.11614}{\mathbf{5.12}} & \ORcell{1.8119}{1.8119}{1.04} & \ORcell{1.05999}{1.05999}{\mathbf{65.99}} & \ORcell{0.3964}{0.3964}{0.2602} & \ORcell{0.29444}{0.29444}{27.51} & \ORcell{0.92967}{0.92967}{\mathbf{63.63}} \\\\ \ORcell{0}{4.78521}{\text{− local drift}} & \ORcell{0.10142}{0.10142}{12.60} & \ORcell{0.53108}{0.53108}{7.94} & \ORcell{0.24973}{0.24973}{8.86} & \ORcell{1.8119}{1.8119}{1.02} & \ORcell{1.2312}{1.2312}{63.54} & \ORcell{0.3964}{0.3964}{0.2643} & \ORcell{0.29444}{0.29444}{25.13} & \ORcell{1.10088}{1.10088}{62.20} \\\\ \ORcell{0}{3.11998}{\text{− common drift}} & \ORcell{0.35115}{0.35115}{9.98} & \ORcell{0.53108}{0.53108}{5.26} & \ORcell{0.24973}{0.24973}{6.04} & \ORcell{1.67831}{1.67831}{\mathbf{0.84}} & \ORcell{1.2312}{1.2312}{61.77} & \ORcell{0.18757}{0.18757}{\mathbf{0.2662}} & \ORcell{0.12323}{0.12323}{\mathbf{24.35}} & \ORcell{1.10088}{1.10088}{60.13} \\\\ \ORcell{0}{2.08508}{\text{− both drift terms}} & \ORcell{0.35115}{0.35115}{2.24} & \ORcell{0.53108}{0.53108}{2.36} & \ORcell{0.24973}{0.24973}{2.46} & \ORcell{1.8119}{1.8119}{0.10} & \ORcell{1.2312}{1.2312}{73.96} & \ORcell{0.3964}{0.3964}{0.2621} & \ORcell{0.29444}{0.29444}{30.18} & \ORcell{1.10088}{1.10088}{40.98} \\\\ \ORcell{0}{3.18541}{\text{− benign replay}} & \ORcell{0.10142}{0.10142}{13.98} & \ORcell{0.53108}{0.53108}{8.94} & \ORcell{0}{0}{10.12} & \ORcell{1.8119}{1.8119}{1.38} & \ORcell{1.2312}{1.2312}{62.07} & \ORcell{0.3964}{0.3964}{0.2483} & \ORcell{0.29444}{0.29444}{29.43} & \ORcell{1.10088}{1.10088}{62.49} \\\\ \hline \end{array}$$
AuthorsOpenReview ↗

Official Comment by Authors(W2 - Part2/2)

The individual removals support both drift scales and benign replay. Removing local drift weakens safety and reduces TIFA and MUSIQ. Removing common drift gives the largest TIFA reduction among the single removals, from 63.63% to 60.13%, and lowers MUSIQ from 65.99 to 61.77. This directly supports common drift’s contribution to retaining requested benign attributes and perceptual quality. Removing replay also lowers TIFA and MUSIQ while worsening safety and COCO FID.

Removing both drift terms reduces TIFA to 40.98%, and the displayed I2P outputs collapse into recurring flat, colorful illustrated portraits, substantially impairing the preservation of requested benign attributes. Together with the individual removals, this result supports the complementary roles of local and common drift in maintaining content and generation quality during safety alignment. The collapsed variant is excluded from ranking. All loss-removal controls use 14,790 updates.

Qualitative examples are shown in Figure 9.

We have revised the manuscript to include these component ablations and the attribute evaluation protocol in Section 5.3, Table 4, and Appendices C.11 and C.13.

References

  • [1] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna and Noah A Smith, TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering, ICCV 2023
  • [2] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar and Feng Yang, MUSIQ: Multi-Scale Image Quality Transformer, ICCV 2021
Reviewer_kEpVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Safety Transition Preference Optimization (SaTPO), for safe alignment of text-to-image diffusion models. The core idea is: Instead of doing preference optimization on the complete generated image, SaTPO constructs unsafe prompt and safe rewrite as one-step transition preference pair with shared parent state and shared noise. Thus safety alignment is performed on the denoising intermediate step. In addition, multi-scale drift constraints and benign replay are combined to retain the generation quality ability of the model. The effectiveness of the above methods is verified on SD1.5 and SDXL.

Critical strengths
  1. The problem formulation is clear, and reasonable improvements are proposed for the limitations of image-level preference. The complete-image-level preference optimization compresses the complex changes in multiple denoising steps into coarse-grained scalar signals, which is easy to entwining security-related attributes with unrelated factors such as composition, semantics, and random drift. This problem analysis is reasonable and important because secure alignment does require a balance between suppressing harmful content and preserving ordinary generation capabilities.

  2. The preference solution design for shared parent state and noise is practical. The SaTPO uses the natural pairing relationship between unsafe prompt and safe rewrite to construct one-step transitions under the same parent state and shared noise. This design avoids relying on additional noisy latent reward models or global image evaluation, and is more concise than stepwise methods that require reward model ranking.

  3. While ensuring safety alignment, this paper also takes into account image quality. The trade-off between safety and quality is achieved by drift constraints and benign replay methods.

Critical weaknesses
  1. The construction of preference pairs is reasonable, but drift penalty and benign replay are the reuse of existing schemes, lacking in-depth exploration of the trade-off between step-wise transitions, such as adaptive adjustment of weights for adaptation.

  2. Some ablation experiments are missing, such as showing the impact of drift penalty on generation quality and safety alignment.

  3. This paper proposes a method using post-training to improve safety alignment, but lacks a deeper comparison with other stepwise preference optimization methods in this task.

  4. The experiments are conducted on SD-v1.5 and SDXL, both within the Stable Diffusion family, leaving the generalization validation of other diffusion model architectures, such as FLUX or QwenImage.

Questions for authors
  1. The paper introduces multi-scale drift regularization, could the authors provide an ablation that removes or separately disables these drift constraints, in order to quantify their individual contribution to safety metrics and benign-generation quality?

  2. The paper emphasizes the use of shared parent states and shared noise for coupled one-step transition preferences. Could the authors provide a more direct comparison with other step-wise preference optimization methods, such as D3PO, SPO, or Diffusion-SDPO, under a comparable post-training setup?

9 comments
AuthorsOpenReview ↗

Official Comment by Authors(W1)

We sincerely thank you for the constructive feedback. Below, we address the raised concerns, with details in the revised version.


W1 — Trade-off and adaptive loss weighting

“The construction of preference pairs is reasonable, but drift penalty and benign replay are the reuse of existing schemes, lacking in-depth exploration of the trade-off between step-wise transitions, such as adaptive adjustment of weights for adaptation.”

SaTPO transfers rewrite-induced directions to transitions under the original unsafe condition, providing incremental safety supervision. However, because the preference margin can increase even when the preferred transition becomes less likely, provided that the rejected transition's log-probability decreases more. Moreover, equal shifts in both candidates' reference-relative log ratios leave the unclipped preference margin invariant, allowing small unidirectional shifts to accumulate over successive denoising steps. We therefore introduce multi-scale drift regularization. Local drift penalizes deviations in each candidate's reference-relative log ratio at individual steps. Common drift penalizes the signed sum, across the trajectory, of the two candidates' mean log ratio at each step. This constrains cumulative shifts that the stepwise preference margin leaves unconstrained. Benign replay complements these penalties by anchoring transitions under ordinary prompts to the reference model. Together, these terms allow SaTPO to make local safety corrections while limiting drift at the transition and trajectory levels.

From an optimization perspective, SaTPO already investigates step-wise trade-offs by incorporating timestep-dependent weighting (w(t)w(t)), which theoretically and empirically prioritizes early denoising steps where safety separation peaks (Figure 2(b)). To rigorously evaluate the trade-off among these terms, we have added component ablations in Section 5.3 and Table 4 (detailed in our W2 & Q1 response below), quantifying how local drift, common drift, and benign replay balance safety, TIFA (benign attribute retention), MUSIQ (perceptual quality), and COCO metrics.

We keep the loss coefficients fixed during training (Appendix B.2) to simplify optimization and avoid additional hyperparameters. However, we fully agree that dynamically adapting these weights during optimization is a valuable direction for future work to further optimize step-wise trade-offs. For example, adjusting these coefficients based on running safety margins or drift magnitudes.

AuthorsOpenReview ↗

Official Comment by Authors(W2&Q1 - Part1/2)

W2 & Q1 — Individual drift components

“Some ablation experiments are missing, such as showing the impact of drift penalty on generation quality and safety alignment.”

“Could the authors provide an ablation that removes or separately disables these drift constraints, in order to quantify their individual contribution to safety metrics and benign-generation quality?”

We have added separate and joint removals of the two drift terms, retaining benign replay and fixing the data and update budget across the three controls. Following reviewer itda's suggestion, we also assess benign attribute retention with TIFA [1], which asks visual questions about the benign content explicitly requested in unsafe prompts. MUSIQ [2] complements this evaluation by assessing perceptual image quality without a reference image, including sensitivity to blur and other visual distortions.

Local drift constrains each candidate's transition log ratio, while common drift constrains their accumulated signed shift across steps. The table therefore examines both individual transitions and their accumulated change, with a separate replay removal for context. Unsafe rates and TIFA are reported as percentages.

\begin{array}{lllllllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SDXL configuration}}}}\hspace{10.04262em}\vphantom{\textbf{PV}} & \rlap{\hspace{8.62492em}\textbf{I2P-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{21.22644em}{0.04em}}}\hspace{2.47767em}\vphantom{\textbf{PV}} & \hspace{2.83754em} & \hspace{2.27483em} & \hspace{5.39917em} & \hspace{4.73723em} & \rlap{\hspace{0.67994em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{6.93081em}{0.04em}}}\hspace{3.5671em}\vphantom{\textbf{PV}} & \hspace{2.86371em} & \rlap{\hspace{0em}\textbf{I2P-1000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{3.97659em}{0.04em}}}\hspace{4.47659em}\vphantom{\textbf{PV}} \\\\ \hspace{10.04262em} & \ORcell{0}{0}{\textbf{PV ↓}} & \ORcell{0}{0}{\textbf{Q16 ↓}} & \ORcell{0.11505}{0.11505}{\textbf{IP ↓}} & \ORcell{0}{0}{\textbf{NudeNet ↓}} & \ORcell{0}{0}{\textbf{MUSIQ ↑}} & \ORcell{0}{0}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} & \ORcell{0.41331}{0.41331}{\textbf{TIFA ↑}} \\\\ \hline \ORcell{0}{6.80813}{\text{SaTPO}} & \ORcell{0.21756}{0.21756}{\mathbf{9.30}} & \ORcell{0.3975}{0.3975}{\mathbf{4.10}} & \ORcell{0.11614}{0.11614}{\mathbf{5.12}} & \ORcell{1.8119}{1.8119}{1.04} & \ORcell{1.05999}{1.05999}{\mathbf{65.99}} & \ORcell{0.3964}{0.3964}{0.2602} & \ORcell{0.29444}{0.29444}{27.51} & \ORcell{0.92967}{0.92967}{\mathbf{63.63}} \\\\ \ORcell{0}{4.78521}{\text{− local drift}} & \ORcell{0.10142}{0.10142}{12.60} & \ORcell{0.53108}{0.53108}{7.94} & \ORcell{0.24973}{0.24973}{8.86} & \ORcell{1.8119}{1.8119}{1.02} & \ORcell{1.2312}{1.2312}{63.54} & \ORcell{0.3964}{0.3964}{0.2643} & \ORcell{0.29444}{0.29444}{25.13} & \ORcell{1.10088}{1.10088}{62.20} \\\\ \ORcell{0}{3.11998}{\text{− common drift}} & \ORcell{0.35115}{0.35115}{9.98} & \ORcell{0.53108}{0.53108}{5.26} & \ORcell{0.24973}{0.24973}{6.04} & \ORcell{1.67831}{1.67831}{\mathbf{0.84}} & \ORcell{1.2312}{1.2312}{61.77} & \ORcell{0.18757}{0.18757}{\mathbf{0.2662}} & \ORcell{0.12323}{0.12323}{\mathbf{24.35}} & \ORcell{1.10088}{1.10088}{60.13} \\\\ \ORcell{0}{2.08508}{\text{− both drift terms}} & \ORcell{0.35115}{0.35115}{2.24} & \ORcell{0.53108}{0.53108}{2.36} & \ORcell{0.24973}{0.24973}{2.46} & \ORcell{1.8119}{1.8119}{0.10} & \ORcell{1.2312}{1.2312}{73.96} & \ORcell{0.3964}{0.3964}{0.2621} & \ORcell{0.29444}{0.29444}{30.18} & \ORcell{1.10088}{1.10088}{40.98} \\\\ \ORcell{0}{3.18541}{\text{− benign replay}} & \ORcell{0.10142}{0.10142}{13.98} & \ORcell{0.53108}{0.53108}{8.94} & \ORcell{0}{0}{10.12} & \ORcell{1.8119}{1.8119}{1.38} & \ORcell{1.2312}{1.2312}{62.07} & \ORcell{0.3964}{0.3964}{0.2483} & \ORcell{0.29444}{0.29444}{29.43} & \ORcell{1.10088}{1.10088}{62.49} \\\\ \hline \end{array}$$
AuthorsOpenReview ↗

Official Comment by Authors(W2&Q1 - Part2/2)

The individual removals support both drift scales and benign replay. Removing local drift weakens safety and reduces TIFA and MUSIQ. Removing common drift gives the largest TIFA reduction among the single removals, from 63.63% to 60.13%, and lowers MUSIQ from 65.99 to 61.77. This directly supports common drift’s contribution to retaining requested benign attributes and perceptual quality. Removing replay also lowers TIFA and MUSIQ while worsening safety and COCO FID.

Removing both drift terms reduces TIFA to 40.98%, and the displayed I2P outputs collapse into recurring flat, colorful illustrated portraits, substantially impairing the preservation of requested benign attributes. Together with the individual removals, this result supports the complementary roles of local and common drift in maintaining content and generation quality during safety alignment. The collapsed variant is excluded from ranking. All loss-removal controls use 14,790 updates.

Qualitative examples are shown in Figure 9.

We have revised the manuscript to include these component ablations and the attribute evaluation protocol in Section 5.3, Table 4, and Appendices C.11 and C.13.

References

  • [1] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna and Noah A Smith, TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering, ICCV 2023
  • [2] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar and Feng Yang, MUSIQ: Multi-Scale Image Quality Transformer, ICCV 2021
Reviewer_6DgeOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

SaTPO aligns text-to-image diffusion models through paired one-step transitions. A frozen reference generates candidates from a shared parent latent and noise under an unsafe prompt and its safe rewrite. Both candidates are scored under the unsafe prompt, and a clipped preference objective favors the rewrite-conditioned candidate. Local and cross-step drift penalties constrain reference-relative log-ratios, while benign replay aims to preserve ordinary generation. The paper also introduces CoProPV, a backbone-specific refinement of AlignGuard’s CoProV2 through image regeneration and PerspectiveVision filtering. Experiments on SD-v1.5 and SDXL assess safety, benign generation, and robustness to fixed adversarial prompts.

Critical strengths
  1. Clear, interpretable preference construction: Shared parents and noise isolate the rewrite-induced candidate displacement (Eq. 4). SaTPO assigns preferences by prompt role rather than LPO’s learned latent-reward ranking (Zhang et al., 2025), and requires no safe rewrite at inference. A matched parent-sharing ablation supports this component: SDXL I2P PV/IP fall from 12.40/8.20 to 9.80/5.74 at the same 8,839-step budget (Table 5(b); Appendix C.9).

  2. Promising safety–quality trade-offs across two backbones: Among the methods in Table 1, SaTPO achieves the lowest PV, Q16, and IP rates on I2P and UD for both backbones. Relative to AlignGuard, COCO FID improves from 32.28 to 25.95 on SD-v1.5 and from 36.50 to 27.51 on SDXL (Figure 5(b)). Removing benign replay further lowers I2P detection rates but substantially degrades benign generation, supporting its role in preserving quality (Table 4(a)).

Critical weaknesses
  1. The drift penalties and timestep schedule are insufficiently isolated: No local-only, common-only, or no-drift ablations retaining benign replay are reported, leaving the penalties’ individual benefits and complementarity unclear. Table 4(b) compares the proposed schedule only with a reversal that changes both weights and their nonzero support. This supports earlier updates for safety relative to that reversal, but does not establish an advantage over uniform weighting. Early-step emphasis also has precedent in Kang et al. (2026), although their timestep sampling and reward scaling differ from SaTPO’s loss weighting.

  2. Data and training-budget differences limit attribution: AlignGuard and SafetyBPO use released checkpoints, whereas SaTPO trains on the newly qualified CoProPV data (§5.1; Appendix C.2). SDXL LPO and SaTPO each train for one epoch on the same 3,944 CoProPV prompt pairs, but LPO’s native filtering yields 540 policy optimizer updates, versus SaTPO’s 14,809 recorded training steps (Appendix C.8; Table 6). These comparisons establish performance under the reported configurations, but do not isolate optimization gains from data qualification and training-budget effects. The matched parent-sharing ablation isolates that component within SaTPO.

References

  • Liu, R., et al. AlignGuard: Scalable Safety Alignment for Text-to-Image Generation. ICCV, 2025.
  • Zhang, T., et al. Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization. NeurIPS, 2025.
  • Kang, J., Lim, S., Baek, K., and Shim, H. Rethinking Direct Preference Optimization in Diffusion Models. AAAI, 2026.
Questions for authors
  1. Does the gap to adapted SDXL LPO persist under comparable update or compute budgets? Appendix C.8 and Tables 9 and 13 provide additional LPO results, but Tables 3 and 12 omit LPO from the fixed adversarial evaluation. Could you explain the omission and provide SDXL results if available?
  2. Could you report local-only, common-only, and no-drift controls while retaining benign replay and fixing the data and training budget?
  3. Could you compare the proposed schedule with uniform weighting under a fixed training budget, including a uniform control over the same positive-weight timesteps to separate weighting-profile effects from timestep support?
7 comments
AuthorsOpenReview ↗

Official Comment by Authors(W1&Q2&Q3 - Part1/3)

We sincerely thank you for the positive feedback and constructive suggestions. Below, we address the remaining concerns, with details in the revised version.


W1 & Q2 & Q3 — Drift components and timestep weighting

“Could you report local-only, common-only, and no-drift controls while retaining benign replay and fixing the data and training budget?”

We have conducted a comprehensive ablation study of SaTPO. Following reviewer itda's suggestion, we also assess benign attribute retention with TIFA [1], which asks visual questions about the benign content explicitly requested in unsafe prompts. MUSIQ [2] complements this evaluation by assessing perceptual image quality without a reference image, including sensitivity to blur and other visual distortions.

Local drift constrains each candidate's transition log ratio, while common drift constrains their accumulated signed shift across steps. The table therefore examines both individual transitions and their accumulated change, with a separate replay removal for context. Unsafe rates and TIFA are reported as percentages.

\begin{array}{lllllllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SDXL configuration}}}}\hspace{10.04262em}\vphantom{\textbf{PV}} & \rlap{\hspace{8.62492em}\textbf{I2P-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{21.22644em}{0.04em}}}\hspace{2.47767em}\vphantom{\textbf{PV}} & \hspace{2.83754em} & \hspace{2.27483em} & \hspace{5.39917em} & \hspace{4.73723em} & \rlap{\hspace{0.67994em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{6.93081em}{0.04em}}}\hspace{3.5671em}\vphantom{\textbf{PV}} & \hspace{2.86371em} & \rlap{\hspace{0em}\textbf{I2P-1000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{3.97659em}{0.04em}}}\hspace{4.47659em}\vphantom{\textbf{PV}} \\\\ \hspace{10.04262em} & \ORcell{0}{0}{\textbf{PV ↓}} & \ORcell{0}{0}{\textbf{Q16 ↓}} & \ORcell{0.11505}{0.11505}{\textbf{IP ↓}} & \ORcell{0}{0}{\textbf{NudeNet ↓}} & \ORcell{0}{0}{\textbf{MUSIQ ↑}} & \ORcell{0}{0}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} & \ORcell{0.41331}{0.41331}{\textbf{TIFA ↑}} \\\\ \hline \ORcell{0}{6.80813}{\text{SaTPO}} & \ORcell{0.21756}{0.21756}{\mathbf{9.30}} & \ORcell{0.3975}{0.3975}{\mathbf{4.10}} & \ORcell{0.11614}{0.11614}{\mathbf{5.12}} & \ORcell{1.8119}{1.8119}{1.04} & \ORcell{1.05999}{1.05999}{\mathbf{65.99}} & \ORcell{0.3964}{0.3964}{0.2602} & \ORcell{0.29444}{0.29444}{27.51} & \ORcell{0.92967}{0.92967}{\mathbf{63.63}} \\\\ \ORcell{0}{4.78521}{\text{− local drift}} & \ORcell{0.10142}{0.10142}{12.60} & \ORcell{0.53108}{0.53108}{7.94} & \ORcell{0.24973}{0.24973}{8.86} & \ORcell{1.8119}{1.8119}{1.02} & \ORcell{1.2312}{1.2312}{63.54} & \ORcell{0.3964}{0.3964}{0.2643} & \ORcell{0.29444}{0.29444}{25.13} & \ORcell{1.10088}{1.10088}{62.20} \\\\ \ORcell{0}{3.11998}{\text{− common drift}} & \ORcell{0.35115}{0.35115}{9.98} & \ORcell{0.53108}{0.53108}{5.26} & \ORcell{0.24973}{0.24973}{6.04} & \ORcell{1.67831}{1.67831}{\mathbf{0.84}} & \ORcell{1.2312}{1.2312}{61.77} & \ORcell{0.18757}{0.18757}{\mathbf{0.2662}} & \ORcell{0.12323}{0.12323}{\mathbf{24.35}} & \ORcell{1.10088}{1.10088}{60.13} \\\\ \ORcell{0}{2.08508}{\text{− both drift terms}} & \ORcell{0.35115}{0.35115}{2.24} & \ORcell{0.53108}{0.53108}{2.36} & \ORcell{0.24973}{0.24973}{2.46} & \ORcell{1.8119}{1.8119}{0.10} & \ORcell{1.2312}{1.2312}{73.96} & \ORcell{0.3964}{0.3964}{0.2621} & \ORcell{0.29444}{0.29444}{30.18} & \ORcell{1.10088}{1.10088}{40.98} \\\\ \ORcell{0}{3.18541}{\text{− benign replay}} & \ORcell{0.10142}{0.10142}{13.98} & \ORcell{0.53108}{0.53108}{8.94} & \ORcell{0}{0}{10.12} & \ORcell{1.8119}{1.8119}{1.38} & \ORcell{1.2312}{1.2312}{62.07} & \ORcell{0.3964}{0.3964}{0.2483} & \ORcell{0.29444}{0.29444}{29.43} & \ORcell{1.10088}{1.10088}{62.49} \\\\ \hline \end{array}$$
AuthorsOpenReview ↗

Official Comment by Authors(W1&Q2&Q3 - Part2/3)

The individual removals support both drift scales and benign replay. Removing local drift weakens safety and reduces TIFA and MUSIQ. Removing common drift gives the largest TIFA reduction among the single removals, from 63.63% to 60.13%, and lowers MUSIQ from 65.99 to 61.77. This directly supports common drift’s contribution to retaining requested benign attributes and perceptual quality. Removing replay also lowers TIFA and MUSIQ while worsening safety and COCO FID.

Removing both drift terms reduces TIFA to 40.98%, and the displayed I2P outputs collapse into recurring flat, colorful illustrated portraits, substantially impairing the preservation of requested benign attributes. Together with the individual removals, this result supports the complementary roles of local and common drift in maintaining content and generation quality during safety alignment. The collapsed variant is excluded from ranking. All loss-removal controls use 14,790 updates.

Qualitative examples are shown in Figure 9.

We have revised the manuscript to include these component ablations and the attribute evaluation protocol in Section 5.3, Table 4, and Appendices C.11 and C.13.

AuthorsOpenReview ↗

Official Comment by Authors(W1&Q2&Q3 - Part3/3)

“Could you compare the proposed schedule with uniform weighting under a fixed training budget, including a uniform control over the same positive-weight timesteps to separate weighting-profile effects from timestep support? Early-step emphasis also has precedent in Kang et al. (2026), although their timestep sampling and reward scaling differ from SaTPO's loss weighting.”

We have added a Uniform control using exactly the same training pairs and positive weighting timesteps as SaTPO. The proposed schedule achieves lower unsafe rates on all four detectors, as shown below.

\begin{array}{lllllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SD-v1.5 schedule}}}}\hspace{8.47554em}\vphantom{\textbf{PV}} & \rlap{\hspace{3.34026em}\textbf{I2P-5000 (\\%, ↓)}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{13.89923em}{0.04em}}}\hspace{2.27483em}\vphantom{\textbf{PV}} & \hspace{2.27483em} & \hspace{2.27483em} & \hspace{4.57474em} & \rlap{\hspace{0.77972em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{7.13037em}{0.04em}}}\hspace{3.76666em}\vphantom{\textbf{PV}} & \hspace{2.86371em} \\\\ \hspace{8.47554em} & \ORcell{0.3108}{0.3108}{\textbf{PV}} & \ORcell{0.13086}{0.13086}{\textbf{Q16}} & \ORcell{0.52727}{0.52727}{\textbf{IP}} & \ORcell{0}{0}{\textbf{NudeNet}} & \ORcell{0.09978}{0.09978}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} \\\\ \hline \ORcell{0}{5.24105}{\text{SaTPO}} & \ORcell{0.11614}{0.11614}{\mathbf{6.00}} & \ORcell{0.11614}{0.11614}{\mathbf{4.76}} & \ORcell{0.11614}{0.11614}{\mathbf{5.36}} & \ORcell{1.2661}{1.2661}{\mathbf{0.68}} & \ORcell{0.24646}{0.24646}{0.25727} & \ORcell{0.29444}{0.29444}{25.95} \\\\ \ORcell{0}{4.79939}{\text{Uniform}} & \ORcell{0.24973}{0.24973}{7.92} & \ORcell{0.24973}{0.24973}{5.28} & \ORcell{0.24973}{0.24973}{5.98} & \ORcell{1.39969}{1.39969}{0.74} & \ORcell{0.24646}{0.24646}{0.25630} & \ORcell{0.29444}{0.29444}{25.21} \\\\ \ORcell{0}{5.09601}{\text{Reverse}} & \ORcell{0}{0}{12.16} & \ORcell{0}{0}{14.14} & \ORcell{0}{0}{15.94} & \ORcell{1.39969}{1.39969}{2.14} & \ORcell{0}{0}{\mathbf{0.26134}} & \ORcell{0.12323}{0.12323}{\mathbf{24.80}} \\\\ \hline \end{array}$$ **The schedule places more weight where local preferences carry a stronger safety signal.** At larger t, denoising has more scope to alter semantic structure, while later steps increasingly refine existing content. Figure 2(b) supports this motivation, showing a larger mean SafetyLRM gap earlier in denoising and a smaller gap toward later steps. The smooth schedule $w(t)=3u_t^2-2u_t^3$, $u_t=\mathrm{clip}((t-250)/250,0,1)$ emphasizes this part of the trajectory. The controlled Uniform comparison therefore supports a specific design choice, allocating more of the preference loss to stages with stronger observed safety separation. Kang et al. \[3] also emphasized earlier steps through timestep sampling and reward scaling. SaTPO applies loss weighting to local preferences constructed from prompt counterfactuals, which are the central contribution. We have revised the manuscript to include the schedule comparison in Table 5 (p. 10), with analysis in Section 5.3 (p. 9) and protocol details in Appendix C.11 (p. 24). **References** * \[1] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna and Noah A Smith, TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering, ICCV 2023 * \[2] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar and Feng Yang, MUSIQ: Multi-Scale Image Quality Transformer, ICCV 2021 * \[3] Junyong Kang, Seohyun Lim, Kyungjune Baek and Hyunjung Shim, Rethinking Direct Preference Optimization in Diffusion Models, AAAI 2026
Reviewer_itdaOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the safety alignment of text-to-image diffusion models. As a preference optimization method, this paper proposes SaTPO, a step-wise alignment instead of image-level one, which provides fine-grained preference supervision over denoising transitions. Together with multi-scale drift regularization and benign replay, the proposed optimization preserves the normal generation capability. To support step-wise optimization, the authors also propose a paired safe-unsafe generation preference dataset. Experiments on SD 1.5 and SDXL verifies that SaTPO further lowers the inappropriate generation rate.

Critical strengths
  1. Meaningful problem and interesting formulation. The paper addresses an important challenge in text-to-image safety alignment. SaTPO constructs step-level safety preferences using unsafe prompts and safe rewrites under shared latent states and noise, providing directional supervision without requiring a learned reward model.
  2. Improved empirical performance. SaTPO achieves improved unsafe-content detection rates and better FID than representative preference-based safety alignment baselines across SD-v1.5 and SDXL.
Critical weaknesses
  1. Insufficient validation of the semantic preservation claim. The paper argues that step-level preferences better preserve task-relevant semantics than image-level preferences. However, the evaluation mainly relies on FID and CLIP scores on benign prompts, without directly measuring the preservation of benign attributes under unsafe prompts. Moreover, the improvements are not clearly isolated from the effects of drift regularization and benign replay.
  2. Limited evaluation on modern diffusion architectures. Experiments are restricted to U-Net-based SD-v1.5 and SDXL, without evaluating recent DiT-based models such as FLUX or SD3. The generalizability of the proposed transition-level optimization to modern diffusion architectures and sampling paradigms therefore remains unclear.
Questions for authors

How sensitive is SaTPO to the quality of safe prompt rewrites? Since shared-state coupling isolates rewrite-induced differences rather than safety-specific differences, how does the method ensure that unrelated semantic changes introduced by safe rewrites do not become part of the learned preference direction?

9 comments
AuthorsOpenReview ↗

Official Comment by Authors(W1 - Part1/2)

We sincerely thank you for the constructive feedback. Below, we address the raised concerns, with details in revised version.


W1 — Benign attributes under unsafe prompts and component attribution

“However, the evaluation mainly relies on FID and CLIP scores on benign prompts, without directly measuring the preservation of benign attributes under unsafe prompts. Moreover, the improvements are not clearly isolated from the effects of drift regularization and benign replay.”

We have added an evaluation of benign attributes preservation under unsafe prompts, along with component ablations. Benign attribute preservation on I2P-1000. We adapt TIFA [1] to evaluate the benign content explicitly requested in the original unsafe prompts. First, Qwen extracts explicit, visually testable benign attributes from the prompt text, excluding harmful details and attributes that depend on them. The TIFA question generator converts these attributes into multiple choice questions, and UnifiedQA checks consistency between the questions and expected answers. BLIP answers the same frozen questions for every model, with correctness averaged within each prompt and then across prompts, so the score measures retention of requested benign content. We compare full SaTPO against AlignGuard [2] (the strong baseline on I2P-5000) and component ablations. As shown below, full SaTPO achieves the highest attribute score, and each single-component ablation still scores above AlignGuard. This confirms that both the core preference alignment and the regularization component could effectively preserve benign attributes.

\hline \textbf{SDXL method / ablation} & \textbf{TIFA for benign attributes (\\%) ↑} \\\\ \hline \text{SaTPO} & \mathbf{63.63} \\\\ \text{− local drift} & 62.20 \\\\ \text{− common drift} & 60.13 \\\\ \text{− benign replay} & 62.49 \\\\ \text{AlignGuard} & 58.15 \\\\ \hline \end{array}$$
AuthorsOpenReview ↗

Official Comment by Authors(W1 - Part2/2)

Supplementary ablation study. We further evaluate unsafe-content detection rates on I2P-5000 and CLIP/FID scores on COCO-5000. As shown below, all three regularization terms contribute to safety alignment performance. In particular, common drift plays a critical role in preserving benign attributes under unsafe prompts, while benign replay helps maintain overall visual fidelity on COCO-5000. Overall, the complete SaTPO method achieves the best balance across safety, image quality, and benign attribute retention.

\begin{array}{llllllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SDXL configuration}}}}\hspace{10.04262em}\vphantom{\textbf{PV}} & \rlap{\hspace{3.22412em}\textbf{I2P-5000 (\\%, ↓)}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{13.66695em}{0.04em}}}\hspace{2.27483em}\vphantom{\textbf{PV}} & \hspace{2.04255em} & \hspace{2.27483em} & \hspace{4.57474em} & \rlap{\hspace{0.67994em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{6.93081em}{0.04em}}}\hspace{3.5671em}\vphantom{\textbf{PV}} & \hspace{2.86371em} & \rlap{\hspace{0em}\textbf{I2P-1000 (\\%, ↑)}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{7.21872em}{0.04em}}}\hspace{7.71872em}\vphantom{\textbf{PV}} \\\\ \hspace{10.04262em} & \ORcell{0.3108}{0.3108}{\textbf{PV}} & \ORcell{0.01472}{0.01472}{\textbf{Q16}} & \ORcell{0.52727}{0.52727}{\textbf{IP}} & \ORcell{0}{0}{\textbf{NudeNet}} & \ORcell{0}{0}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} & \ORcell{2.44659}{2.44659}{\textbf{TIFA}} \\\\ \hline \ORcell{0}{6.80813}{\text{SaTPO}} & \ORcell{0.11614}{0.11614}{\mathbf{9.30}} & \ORcell{0}{0}{\mathbf{4.10}} & \ORcell{0.11614}{0.11614}{\mathbf{5.12}} & \ORcell{1.39969}{1.39969}{1.04} & \ORcell{0.3964}{0.3964}{0.2602} & \ORcell{0.29444}{0.29444}{27.51} & \ORcell{2.55073}{2.55073}{\mathbf{63.63}} \\\\ \ORcell{0}{4.78521}{\text{− local drift}} & \ORcell{0}{0}{12.60} & \ORcell{0.13359}{0.13359}{7.94} & \ORcell{0.24973}{0.24973}{8.86} & \ORcell{1.39969}{1.39969}{1.02} & \ORcell{0.3964}{0.3964}{0.2643} & \ORcell{0.29444}{0.29444}{25.13} & \ORcell{2.72194}{2.72194}{62.20} \\\\ \ORcell{0}{3.11998}{\text{− common drift}} & \ORcell{0.24973}{0.24973}{9.98} & \ORcell{0.13359}{0.13359}{5.26} & \ORcell{0.24973}{0.24973}{6.04} & \ORcell{1.2661}{1.2661}{\mathbf{0.84}} & \ORcell{0.18757}{0.18757}{\mathbf{0.2662}} & \ORcell{0.12323}{0.12323}{\mathbf{24.35}} & \ORcell{2.72194}{2.72194}{60.13} \\\\ \ORcell{0}{3.18541}{\text{− benign replay}} & \ORcell{0}{0}{13.98} & \ORcell{0.13359}{0.13359}{8.94} & \ORcell{0}{0}{10.12} & \ORcell{1.39969}{1.39969}{1.38} & \ORcell{0.3964}{0.3964}{0.2483} & \ORcell{0.29444}{0.29444}{29.43} & \ORcell{2.72194}{2.72194}{62.49} \\\\ \ORcell{0}{4.89972}{\text{AlignGuard}} & \ORcell{0}{0}{15.04} & \ORcell{0.13359}{0.13359}{8.58} & \ORcell{0.24973}{0.24973}{9.42} & \ORcell{1.39969}{1.39969}{0.96} & \ORcell{0.3964}{0.3964}{0.2585} & \ORcell{0.29444}{0.29444}{36.50} & \ORcell{2.72194}{2.72194}{58.15} \\\\ \hline \end{array}$$ Qualitative examples are shown in [Figure 9](https://anonymous.4open.science/r/satpo-rebuttal-figures-92B8/images/figure09_drift_ablations.png). We have revised the manuscript to include the combined component results in Section 5.3 and Table 4. Appendix C.13 describes the attribute evaluation. Matched qualitative examples are shown in [Figure 6](https://anonymous.4open.science/r/satpo-rebuttal-figures-92B8/images/figure06_benign_attributes.png) and [Figure 8](https://anonymous.4open.science/r/satpo-rebuttal-figures-92B8/images/figure08_benign_attributes.png). **References** * \[1] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna and Noah A Smith, TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering, ICCV 2023 * \[2] Runtao Liu, I Chieh Chen, Jindong Gu, Jipeng Zhang, Renjie Pi, Qifeng Chen, Philip Torr, Ashkan Khakzar and Fabio Pizzati, AlignGuard: Scalable Safety Alignment for Text-to-Image Generation, ICCV 2025
AuthorsOpenReview ↗

Official Comment by Authors(W2)

W2 — DiT and flow matching models

“Experiments are restricted to U-Net-based SD-v1.5 and SDXL, without evaluating recent DiT-based models such as FLUX or SD3.”

We extended SaTPO to SD3.5 Medium [1], a flow-matching DiT, and evaluated safety and benign generation quality. Following Flow-GRPO [2], we use Gaussian Flow-SDE transitions with computable densities. Candidate branches share the parent state and noise, and both policy/reference log ratios are evaluated under the original unsafe prompt, preserving SaTPO's local preference construction.

We compare with SHIFT [3], which learns steering directions from contrastive prompts and uses gates to apply them selectively to intermediate activations during inference. We adapt SHIFT to the same seven risk categories and evaluate all configurations using the same prompts, seeds, generation settings, and detectors.

Safety on I2P-5000

\begin{array}{lllll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SD3.5 Medium}}}}\hspace{7.40683em}\vphantom{\textbf{PV}} & \rlap{\hspace{3.34353em}\textbf{I2P-5000 (\\%, ↓)}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{13.90577em}{0.04em}}}\hspace{2.27701em}\vphantom{\textbf{PV}} & \hspace{2.27701em} & \hspace{2.27701em} & \hspace{4.57474em} \\\\ \hspace{7.40683em} & \ORcell{0.31189}{0.31189}{\textbf{PV}} & \ORcell{0.13195}{0.13195}{\textbf{Q16}} & \ORcell{0.52836}{0.52836}{\textbf{IP}} & \ORcell{0}{0}{\textbf{NudeNet}} \\\\ \hline \ORcell{0}{5.3621}{\text{Base}} & \ORcell{0.00109}{0.00109}{38.60} & \ORcell{0.00109}{0.00109}{34.58} & \ORcell{0.00109}{0.00109}{35.62} & \ORcell{1.39969}{1.39969}{1.88} \\\\ \ORcell{0}{4.36754}{\text{SHIFT}} & \ORcell{0}{0}{\underline{18.78}} & \ORcell{0}{0}{\underline{14.62}} & \ORcell{0}{0}{\underline{15.56}} & \ORcell{1.39859}{1.39859}{\underline{1.20}} \\\\ \ORcell{0}{4.17234}{\text{SaTPO}} & \ORcell{0.11723}{0.11723}{\mathbf{7.04}} & \ORcell{0.11723}{0.11723}{\mathbf{2.72}} & \ORcell{0.11723}{0.11723}{\mathbf{3.78}} & \ORcell{1.2661}{1.2661}{\mathbf{1.10}} \\\\ \hline \end{array}$$ SaTPO achieves the lowest detection rate on all four metrics. **Benign generation quality on COCO-5000** $$\def\ORcell#1#2#3{\hspace{#1em}#3\hspace{#2em}} \begin{array}{lll} \hline \rlap{\smash{\raise{-0.65em}{\hspace{0em}\textbf{SD3.5 Medium}}}}\hspace{7.40683em}\vphantom{\textbf{PV}} & \rlap{\hspace{0.77972em}\textbf{COCO-5000}}\rlap{\raise{-0.45em}{\hspace{0.25em}\rule{7.13037em}{0.04em}}}\hspace{3.76666em}\vphantom{\textbf{PV}} & \hspace{2.86371em} \\\\ \hspace{7.40683em} & \ORcell{0.09978}{0.09978}{\textbf{CLIP ↑}} & \ORcell{0}{0}{\textbf{FID ↓}} \\\\ \hline \ORcell{0}{5.3621}{\text{Base}} & \ORcell{0}{0}{\mathbf{0.26594}} & \ORcell{0.12323}{0.12323}{\mathbf{29.08}} \\\\ \ORcell{0}{4.36754}{\text{SHIFT}} & \ORcell{0.24482}{0.24482}{\underline{0.26472}} & \ORcell{0.12323}{0.12323}{\mathbf{29.08}} \\\\ \ORcell{0}{4.17234}{\text{SaTPO}} & \ORcell{0.24646}{0.24646}{0.25057} & \ORcell{0.29335}{0.29335}{\underline{30.76}} \\\\ \hline \end{array}$$ Notably, SHIFT freezes model weights and applies only selective interventions during inference, keeping its generation quality virtually identical to the base model. In comparison, SaTPO incurs a minor changes (−0.0142 CLIP, +1.68 FID). We consider this tradeoff acceptable for safety alignment given the gains across all four detectors. We have revised the manuscript to include the results and protocol in Appendix C.12 and Table 20. Qualitative examples are shown in [Figure 11](https://anonymous.4open.science/r/satpo-rebuttal-figures-92B8/images/figure11_sd35.png). Within the rebuttal period, compatible checkpoints for flow matching backbones were unavailable for the preference optimization baselines used in our original comparison. We are adapting those baselines and will supplement the comparison as their results become available. **References** * \[1] Stability AI, Stable Diffusion 3.5 Medium, model card, 2024 * \[2] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang and Wanli Ouyang, Flow-GRPO: Training Flow Matching Models via Online RL, NeurIPS 2025 * \[3] Nina Konovalova, Ibragim Idrisov and Aibek Alanov, SHIFT: Steering Hidden Intermediates in Flow Transformers, WACV 2027

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.