- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TRACE, a training-free concept erasure method for visual autoregressive text-to-image models. Using paired prompts that respectively include and exclude the target concept, TRACE compares the model's predicted distributions across different generation scales and constructs scale-specific concept subspaces accordingly. It then applies closed-form low-rank weight editing to suppress components of text-projected representations within these subspaces, thereby reducing the generation of target concepts. The paper also introduces TRACE-Lite, which uses a shared concept subspace across scales to reduce editing costs.
Experiments on the Infinity-2B model evaluate concept erasure across three categories: nudity, artistic styles, and copyrighted characters. TRACE is compared with various fine-tuning-based, closed-form editing-based, and inference-time intervention methods. The results demonstrate that TRACE effectively reduces the generation of target concepts while preserving overall image quality, text alignment, and the ability to generate non-target concepts. Although TRACE-Lite adopts a simplified shared-subspace design, it achieves competitive erasure and preservation performance while substantially reducing editing overhead. Ablation studies further support the contributions of unit normalization, response weighting, and scale-specific subspace design to the method's performance.
Critical strengths
-
The method addresses a consequential efficiency limitation. TRACE avoids per-concept gradient-based fine-tuning and the additional conditional forward passes used by inference-time erasure. TRACE-Lite is particularly attractive: its reported editing cost is substantially below S-VARE, while achieving competitive erasure and better COCO preservation metrics. This could make frequent concept updates practically feasible.
-
The scale-aware response measurement is a sensible adaptation to autoregressive generation. Conditioning target and neutral predictions on identical preceding visual tokens controls an important source of variation. The resulting response weights connect subspace construction to the generator’s behavior rather than relying entirely on text-feature geometry. This is the most compelling methodological contribution.
Critical weaknesses
-
Limited evaluation of safety robustness. The safety evaluation is restricted to a filtered nudity-related subset of I2P and uses NudeNet-based detection metrics. Using additional evaluation sets, such as Ring-A-Bell and MMA-Diffusion, would strengthen this evidence.
-
Insufficient coverage of general object-category erasure. The evaluation covers nudity, one artistic style, and one copyrighted character, but does not include systematic erasure of ordinary object categories. While Pikachu provides evidence for entity-specific erasure, it does not establish that the method generalizes to broader categories with substantial variation and semantic overlap.
Questions for authors
-
Can the authors evaluate the methods on additional evaluation sets, such as Ring-A-Bell[1] and MMA-Diffusion[2]? Please report the Attack Success Rate.
-
Can the authors evaluate erasure across several ordinary object categories, such as church, golf ball, garbage truck ,and so on? Please report target-recognition success, preservation of unrelated objects and generation quality.
-
Can the authors add an additional multi-concept test experiment? For example, mix target-neutral prompt pairs from multiple target concepts into a single calibration set, perform joint erasure, and then evaluate the same edited model using held-out prompts for each concept separately. How does this affect attack success, preservation quality, and computational cost?
[1] Tsai Y L, Hsu C Y, Xie C, et al. Ring-a-bell! how reliable are concept removal methods for diffusion models?[C]//International Conference on Learning Representations. 2024, 2024: 41543-41554.
[2] Yang Y, Gao R, Wang X, et al. Mma-diffusion: Multimodal attack on diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024: 7737-7746.