- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targtes at the question of NSFW classifier in Latent Diffusion Models. Traditional Pre-generation methods struggles with the gap between the textual propmt safety and visual image safety, post-generation methods have to wait for the full image finish being denoised which leads to inefficiency. The authors propose FlowGuard, a novel cross-model in-generation frameworks that allows detection within the denoising process. By leveraging linear approximation for latent decoding, curriculum learning and constructing a dataset containing images from different intermediate steps and different underlying model types, FlowGuard achieves the highest ID and OOD performance while reduces the time latency and peak GPU memory.
Critical strengths
- Practical problem targeting: The paper targets cross-model in-generation detection, a setting where existing in-generation detectors (Wukong, IGD) requires a separately trained detector for each backbone. The decomposition proposed in this paper makes adding new backbone to the system inexpensive. On four held-out backbones, the detector is kept frozen and only the linear decoder is fitted, without any safety labels.
- Higher performance with lower inspection overhead: Replacing VAE decoding with the linear projection reduce per-step decoding time from 8.1 s to 0.2 s and peak GPU memory by over 97% at batch size 50, which makes inspecting intermediate states actually affordable.
Critical weaknesses
- Detection is performed only at step 20. At inference, FlowGuard inspects a single fixed step (step 20 of 50) for all backbones and all samples, which is not adaptive. From Fig. 3 it seems detection at earlier steps may already be enough for some backbones, in that case unsafe generations could be terminated earlier and more computation could be saved.
- Baselines in Table 2 are not retrained on the constructed dataset. The post-generation baselines are applied directly to step-20 reconstructions, while FlowGuard is trained on reconstructions from the same dataset. Several baselines label every sample as unsafe (recall 1.000), which indicate that they are evaluated on inputs they were not designed for. So the comparison may be unfair, and it’s still unclear how much of the improvement come from the proposed method itself rather than from training on in-domain data.
Questions for authors
- How was step 20 chosen as the detection step? Is there any possible addaptive mechanism that we could design so that under different conditions and underlying models, we could decide at which specific steps we need to call the detector for a detection. For example, some images may already could be detected at early steps, while some other tricky situations we may need to wait until the later steps
- How would the post-generation baselines in Table 2 perform if they were retrained or fine-tuned on the same step-20 reconstructions used to train FlowGuard?
- Could you add a control trained on the same multi-step data without curriculum ordering, so that the gain of curriculum learning can be separated from the effect of more training data and broader noise-level coverage?
- Appendix C states that FlowGuard is 0.047 below Wukong, while Table 3 reports 0.880 vs. 0.833; which one is correct?