- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper claims that closed-form concept erasure methods become less effective for larger text-to-image diffusion models and attributes this degradation to redundancy in the cross-attention (CA) parameter space. To address this, the paper proposes SPACE, which builds on UCE's objective function by adding an entry-wise L1 norm on the CA weights to encourage sparsity. Due to the non-smoothness of the L1 norm, SPACE uses the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) to iteratively update the CA parameters. The gradient is computed in closed form without backpropagation.
Critical strengths
- The method is sound, fast, storage-efficient, and effective in concept erasure. The paper proves that SPACE will surely be convergent. Also, updating cross-attention weights for 1000 iterations takes less than 2 minutes. Moreover, SPACE achieves over 80% sparsity, which reduces storage requirements significantly after erasure.
- The paper provides thorough analyses demonstrating the importance of cross-attention sparsity. It shows that increasing the erasure strength in UCE does not improve erasure performance and that sparsity, rather than iterative updates, is the key to SPACE's effectiveness.
- The paper is well written and easy to follow.
Critical weaknesses
- The motivation of this paper is not convincing to me. The paper claims that closed-form erasure methods become less effective on larger models due to redundancy in the cross-attention parameter space. Why focus on cross-attention rather than self-attention or feed-forward modules?
- In artistic style erasure, SPACE performs noticeably worse than UCE and SPEED at preserving non-target concepts. This raises the concern that SPACE erases the target concept so hard that it sacrifices the non-target quality. Could lowering the sparsity value or reducing the number of FISTA iterations improve this trade-off?
- The sparsity-based unlearning methods mentioned in line 83, such as SalUn and Scissorhands, are not included in the experiments.
- The paper lacks quantitative results for celebrity erasure; only qualitative results are provided. Also, quantitative results for fictional character erasure using SDXL are also missing.
- Figure 2 is not well designed. It’s full of math formulas and notation. The math formula in the middle is even displayed vertically.
Questions for authors
- How does the number of FISTA iterations affect erasure performance?
- How does affect erasure performance?