- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper targets a gap in concept erasure for text to image models. Erasure methods are usually judged without any cross architecture test. The authors propose Concept Score Relearning (CSR), a parameter level relearning attack. CSR takes a frozen, unerased public reference model, written as phi. At the same latent and the same timestep or noise level it computes the difference between the conditional and the null prediction. The authors evaluate on SD-1.4 and FLUX.1-dev, over six erasure methods (ESD, EA, AC, EAP, CP, MACE) and five concept groups (nudity, violence, artistic style, and the entity, abstraction and relationship categories). They compare against four attack baselines (UnlearnDiffAtk, P4D, Ring-A-Bell, ReFLUX), report CLIPScore, FID and LPIPS for utility, and run a blinded human study with 20 participants. Strict nudity ASR rises from 7.60 percent after erasure to 50.47 percent on FLUX, and from 8.60 percent to 40.29 percent on SD-1.4. Violence rises from 21.33 percent to 71.44 percent on FLUX, and from 39.04 percent to 59.92 percent on SD-1.4. The paper reports first place across the evaluated settings. One concept update takes about 2.4 minutes on SD-1.4 and about 5 minutes on FLUX. The stated conclusion is that suppression after erasure does not imply resistance to later parameter level relearning.
Critical strengths
-
The paper correctly notes that existing parameter level relearning attacks are tied to one architecture. ReFLUX (Jiang et al., 2025) relies on FLUX velocity prediction and on specific attention mechanisms.
-
The study spans six erasure methods, two architectures, five concept groups and four attack baselines. It adds three utility metrics and a human study, all documented in detail. Appendix A.3.6 states the threat model precisely. The attacker holds the erased weights, the target concept prompt, compute, and the unerased reference checkpoint, and does not need to know the erasure algorithm.
Critical weaknesses
-
Liu and Zhang (2025, arXiv 2505.16174) propose a parameter level, concept directed relearning attack on U-Nets. The paper cites it once in related work. At eta equal to 1, CSR is in essence matching the reference model's own conditional prediction, which overlaps strongly with that setting. Since eta equal to 1 is much weaker than eta equal to 3, a reader cannot tell what CSR adds on the SD side beyond that method. The cross architecture story is missing its most important control.
-
Each of the 12 main settings is run once. The appendix states that every cell comes from a single seed 0 attack run. The only error bars in the whole paper come from the human study. The authors themselves write in Appendix A.7.4 that small differences between nearby settings should not be read as statistically meaningful. The default hyperparameters, eta equal to 3 and N equal to 40, were chosen on those same single runs. Two curves show how large the wobble can be. The FLUX anchor sweep gives a lower score at N equal to 10 than at N equal to 5, and the FLUX budget curve moves 21.40, then 17.89, then 22.11, then 38.25. The abstract still says the method consistently ranks first across all evaluated safety settings. The human study reports only means and standard deviations and runs no test at all, while the main text claims the appendix provides a full study design and statistical analysis. On concept reactivation the two methods sit at 4.62 plus or minus 0.91 and 4.52 plus or minus 0.78, so the intervals overlap.
-
ReFLUX is a LoRA fine tune and needs a real target concept image dataset. CSR updates about 2.51 billion parameters on FLUX, which is 21.1 percent of the Transformer, for 200 steps at a learning rate of 1e-4, and it updates the entire U-Net on SD-1.4 for 500 steps at 1e-5. The learning rates differ by a factor of ten and the step counts by a factor of two and a half. The appendix justifies this only by memory and hardware limits. On the SD side, ReFLUX is a port that the authors retuned, and two loss terms calibrated for FLUX were zeroed out. MACE on FLUX keeps only the CFR stage. EAP on FLUX uses an approximate configuration with no direct equivalent. EA on SD-1.4 is reduced from a bi-level optimization to a single prior preservation term. Only ReFLUX shares the white box parameter level threat model with CSR, since the other three baselines attack a frozen victim from prompt space or from a black box. The claim of first place therefore mixes threat models. No baseline uses an unerased reference model as its supervision source, so none of them is matched to CSR on that axis.
-
CLIPScore, FID and LPIPS cover only ESD and EA, and only three concept blocks, namely nudity, violence and Van Gogh. Picasso and the other four erasure methods are absent. The conclusion is still written as preserving general generation utility. The paper has no limitations section.
Questions for authors
-
At eta equal to 1, CSR matches the reference model's conditional prediction. That setting overlaps strongly with the U-Net concept directed noise prediction target of Liu and Zhang (2025, arXiv 2505.16174). Can you run that method as an SD side baseline? If the real insight is the gain from eta equal to 3 over eta equal to 1, from 17.89 to 32.98 percent, then how much of the increment over that baseline comes from the target construction and how much from the extrapolation hyperparameter?
-
The main tables and the ablations are single seed 0 runs, and the paper has no error bars outside the human study. Can you report ASR distributions over at least three attack seeds? The abstract numbers of 50.47 and 40.29 percent are unweighted means over six erasure methods. Can you also give a method weighted view?
-
On FLUX, CSR updates about 2.51 billion parameters, which is 21.1 percent, for 200 steps at 1e-4. On SD-1.4 it updates the whole U-Net for 500 steps at 1e-5. ReFLUX only trains LoRA adapters. How does ReFLUX perform at the same trainable parameter count and the same step count?
-
The appendix says the SD-1.4 port of ReFLUX zeroed two losses that were calibrated for FLUX. Can you report the version with those terms restored, or at least quantify what zeroing them does to ReFLUX itself?
-
The gap between eta equal to 1 and eta equal to 3 suggests that the concept direction is weakened after erasure and needs extrapolation. Can you characterize quantitatively, for instance the cosine similarity or the norm ratio between the post erasure direction and the reference direction as a function of the timestep?
-
CSR needs the unerased reference checkpoint phi. Since an attacker in this position could simply re-distill from phi, what does CSR establish beyond that simpler route?
-
You state that the two prediction spaces are not equivalent and that each direction lives only in its own space, which I accept. Then what exactly is unified?
-
The main text says the appendix provides a full study design and statistical analysis, but Appendix A.8 reports only participant level means and standard deviations and runs no test. The design is paired, so can you add a paired test, an effect size and a sample size justification?
-
Have NudeNet at threshold 0.6, the binary Q16 classifier, WikiArt Top-3, and the 90 percent CLIP decision threshold used for the miscellaneous categories been validated?
-
ReFLUX needs a real target concept image dataset while CSR needs only self generated anchors. Can you make that difference explicit in the main text rather than presenting it only as a capability?