- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The method selects blocks within model, averages theirs activations across all tokens, and computes angles and distances between consecutive steps over denoising steps. These features are then used for classification (safe or unsafe image). Experiments on several diffusion models show better detection than earlier in-generation methods and some savings from stopping early.
Critical strengths
- The idea of watching how the model's internal activations move during denoising (instead of judging a single step) is interesting and seems new to me.
- The method is lightweight and adds almost no cost to generation.
- The experiments cover several model families, and the early-stopping evaluation is close to how such a tool would actually be used.
Critical weaknesses
-
The method averages activations across tokens at selected blocks and computes turning angles between consecutive activation displacements and step lengths during denoising. I struggle to find what meaningful information these features capture or why they should distinguish unsafe from benign generation. Although the authors provide quantitative plots, these results do not sufficiently explain why the features are effective.
-
Benign samples come from COCO captions, while positives come from I2P etc. These differ heavily in prompt style , so activation trajectories could separate them for reasons unrelated to unsafe content.
-
Some important Post-generation baseline is missing, such as Q16 and MHSC.
Questions for authors
- Why the Denoising Dynamics work, what are the extracted feature capture? (W1)
- The shortcut issue in W2.
- Compare with Q16 and the Multi-Headed Safety Classifier.
Response to Reviewer aK3d (1/2)
(W1+Q1) Why the Denoising Dynamics work, what are the extracted feature capture?
Response:
We sincerely apologize for this confusion. In short, the two motion features describe "how much" the activations of the whole layer change and "whether the change keeps going in the same direction". We believe they can separate unsafe from benign generations because the formation of unsafe content often comes with larger adjustments whose direction is less consistent. We would like to explain this in three points:
1. What the features capture. At each denoising step, we average the activations of the same block over spatial positions (or image tokens), which gives one response value per channel. The difference between two consecutive steps is then the change in channel responses (averaged over all positions). Pooling just discards where the change happens, but keeps how the response of the whole layer changes. Step length is the size of this change. Turning angle is how well this change agrees in direction with the previous one.
2. Why they can separate unsafe from benign generations. Prior work shows that semantic attributes in diffusion models form within short time windows (Biroli et al., 2024; Li & Chen, 2024). Such transitions in the sampling dynamics are related to the formation of semantic structure (Ramachandran et al., 2025; Handke et al., 2026). Our hypothesis is that, when a generation moves toward unsafe content, the model makes a larger adjustment, with a change of direction. We propose the step length and turning angle measure this adjustment.
Our observations are consistent with this hypothesis. Specifically, we generate images from 300 I2P prompts with SD 1.4, and manually audit the images. Among these images, 67 contain NSFW content and 233 do not. In the middle of denoising, the median turning angle is 23.3° for the NSFW group and 19.5° for the non-NSFW group. The median step lengths are 0.875 and 0.780, respectively. That is, with the same prompt source, the trajectories that finally produce NSFW content change more in the middle of denoising, and their consecutive updates agree less in direction.
3. Why we need to combine the features and the history. For a single feature, the two groups still overlap clearly, so it is hard to judge a single image from one angle at one step. Therefore, the monitor uses the observations from several layers together with the changes at earlier steps. We show in the manuscript that the two kinds of features are complementary: in the I2P evaluation on SD1.4, the AUC is 0.869 with turning angles only, 0.866 with step length only, and 0.907 with both.