- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Lyapunov-constrained Continuous Concept Erasure (LCCE), a framework for sequential concept erasure in text-to-image diffusion models. The method aims to address two challenges arising from repeated concept erasure: accumulated degradation of non-target generation capabilities and the reappearance of previously erased concepts. Specifically, LCCE introduces a dynamic virtual queue based on Lyapunov optimization to regulate long-term preservation loss, a cumulative preservation bank to maintain semantically related non-target concepts, and a historical retention loss to prevent concept revival. These components are integrated into a closed-form solution for updating cross-attention projections without iterative fine-tuning. Experiments on Stable Diffusion v1.4 demonstrate competitive performance in continuous artist-style, object, and celebrity erasure, with particularly low per-concept editing costs.
Critical strengths
-
Meaningful problem formulation. The paper addresses continuous concept erasure, a practical and important setting where new erasure requests arrive sequentially while previously erased concepts and non-target generation capabilities must be preserved.
-
Well-structured methodology. The proposed framework combines a dynamic virtual queue, a cumulative preservation bank, and a historical retention loss to address different challenges in continuous concept erasure.
-
Promising experimental results. LCCE demonstrates competitive erasure effectiveness and non-target preservation across artist-style, object, and celebrity erasure tasks, particularly over long editing sequences.
Critical weaknesses
-
Concerns about the fairness and completeness of baseline comparisons. This is my major concern. While continuous concept erasure is a meaningful practical problem, several existing methods can efficiently erase all accumulated concepts from scratch whenever a new erasure request arrives, without incurring substantial computational overhead. For example, UCE and SPEED can jointly erase multiple concepts within a short time, while MACE trains concept-specific LoRAs that can be efficiently merged using a closed-form solution. Therefore, these methods are naturally applicable to continuous erasure scenarios, and their computational costs may remain competitive even when newly arriving concepts trigger a fresh multi-concept erasure. I strongly encourage the authors to evaluate these baselines under such a setting and compare both their erasure effectiveness and cumulative computational costs. In particular, comparisons on 100 artistic styles and 100 celebrities would be valuable, given that SPEED and MACE were specifically designed for multi-concept erasure.
-
Insufficient justification of the bounded preservation guarantee. The theoretical analysis in Appendix A establishes that the long-term preservation constraint holds under mean-rate stability of the virtual queue. However, it does not establish that the proposed editing algorithm necessarily satisfies this stability condition. The authors should clarify the assumptions required for the claimed guarantee and provide empirical evidence of queue stability and preservation-budget satisfaction over long erasure sequences.
-
Unclear necessity of Lyapunov-based optimization. The dynamic virtual queue essentially adjusts the weight of the preservation objective across editing stages. Although Table 4 includes an ablation without the dynamic queue, it remains unclear whether Lyapunov optimization offers meaningful advantages over simpler alternatives, such as well-tuned fixed preservation weights or heuristic adaptive weighting. More controlled comparisons are needed to establish the necessity of this design.
-
Incomplete evaluation of long-term erasure effectiveness. The 100-artist experiment in Figure 3 primarily reports CLIP and VQA scores without showing whether previously erased concepts remain suppressed. Moreover, LCCE's VQA score appears to increase substantially as more concepts are erased, which deserves further explanation. The authors should report historical erasure rates throughout the 100-stage process and clarify why the VQA score improves despite successive modifications to the model.
-
Limited experimental validation across model architectures. All experiments are conducted on Stable Diffusion v1.4. Since the proposed approach is intended for practical continuous concept erasure, evaluating it on more recent architectures, such as SDXL or DiT-based models, would help establish its generalizability.
-
Poor figure quality. The figures appear to be embedded as PNG images and become noticeably blurry when enlarged, particularly the plots and their annotations. For academic presentation quality, the authors should use vector graphics for plots and diagrams whenever possible, ensuring that text, curves, and labels remain readable at different scales.
-
Non-standard mathematical notation. The mathematical notation throughout the paper could be improved for clarity and consistency. In particular, vectors and matrices are not consistently distinguished typographically. I suggest adopting a unified convention, such as bold lowercase letters for vectors and bold uppercase letters for matrices, and applying it consistently across the equations.
Questions for authors
See the weakness part
3 comments
Response to Reviewer xmUb
[W3] Necessity of the virtual queue. With solver, bank, retention statistics, and artist order all fixed, the queue outperforms both a tuned fixed weight and a one-step adaptive rule (Appendix G.4). The fixed weight was selected on a separate development set by a rule fixed in advance (Table 20); all three controllers were evaluated once on a confirmation set with new prompts.
| Controller | RR-LLM ↑ | Non-target ΔCLIP ↑ |
|---|---|---|
| Virtual queue | 0.9031 | +0.6594 |
| Fixed weight () | 0.7906 | +0.5607 |
| One-step heuristic | 0.8738 | +0.3425 |
The queue is not just a varying weight: it accumulates preservation debt across all earlier stages. A fixed weight cannot react to accumulated drift, and the one-step rule sees only the last loss. The queue's advantage in erasure over the fixed weight, and in preservation over the one-step rule, comes from this history.
[W4] Long-term erasure and the VQA trend. Early-erased artists stay erased throughout 100 sequential edits (Appendix C.4, Fig. 5). After the first ten edits, RR-CLS and RR-LLM on these artists remain flat through the hundredth edit. UCE and MACE also keep erasure high, but collapse non-target CLIP and VQA (Fig. 3); SPEED preserves quality but leaves styles recognizable. LCCE is the only method that keeps both.
Clarification: the rising VQA score does not indicate returning styles. VQA measures whether the objects named in the prompt are recognizable, not whether a style is present. Abstract styles distort object structure; removing the style makes objects easier to recognize, raising VQA.
[W5] Other architectures. LCCE transfers to SDXL and FLUX.1-schnell without changing the method. Appendix C.8 applies the same closed-form update to both backbones over four sequential artist erasures:
| Backbone | RR-LLM ↑ original → LCCE | CLIP original → LCCE |
|---|---|---|
| SDXL | 0.350 → 0.875 | 29.940 → 30.504 |
| FLUX.1-schnell | 0.825 → 1.000 | 28.698 → 28.449 |
LCCE edits the cross-attention key/value projections of SDXL and the two text-conditioning projections of FLUX. Erasure improves on both architectures; CLIP remains at the original level.
[W6, W7] Figures and notation. Fixed. All plots are now vector PDFs. Vectors use bold lowercase, matrices bold uppercase. The derivations in Appendix A are rewritten and cross-checked.
I remain unconvinced of the practical necessity of the proposed continuous editing setting, as existing editing-based methods can efficiently handle multi-concept erasure with limited additional overhead, while LCCE has not demonstrated a clear advantage over these alternatives (e.g., in the additional experiments, LCCE exhibits a substantially larger non-target CLIP degradation of −2.249 compared to −1.106 for UCE-joint and −0.912 for SPEED-joint); therefore, I maintain my original score.
Response to Reviewer xmUb
Summary. (W1) Against joint re-erasure of all 100 artists, LCCE achieves higher erasure while editing online. (W2) Our guarantee is conditional, as stated; we now verify it empirically. (W3) With everything else fixed, the queue outperforms a tuned fixed weight and a one-step adaptive rule. (W4) Early erasures do not revive over 100 edits; the VQA increase reflects content recognizability, not returning styles.
[W1] Joint re-erasure baselines. LCCE removes more target styles than UCE-joint and SPEED-joint, although they see all 100 targets at once and edit the original weights in a single step (Appendix C.5).
LCCE never revisits the original weights and stores no record of past requests — the defining constraint of the continuous setting (Sec. 3). A deployed model receives a removal request and must be updated in place. Joint re-erasure requires retaining every past request and the original checkpoint, then re-editing and re-validating the entire model at each new request; the CLIP gap reflects this fundamental difference in access.
[W2] Scope of the preservation guarantee. Clarification: the paper does not claim that LCCE guarantees queue stability for every request sequence. As the reviewer notes, Appendix A proves that mean-rate stability implies the long-term budget. The finite-horizon bound (Theorem A.2) holds unconditionally: T1∑t=1TPL(t)≤D+aTZ(T+1)−Z(1). Stability itself requires the standard Lyapunov conditions, now stated explicitly in Appendix A.7: bounded per-stage loss, a budget D that some policy can meet with a nonzero margin, and a per-stage action that minimizes the drift bound up to bounded error.
Empirically, a feasible budget is met by every edited projection (Appendix G.2, Table 18). Values are cumulative loss divided by the budget; below 1 means the budget is satisfied.
The default sets D to the loss of a single edit, which is tighter than a growing preservation bank can sustain. The queue therefore keeps raising preservation pressure, yet the excess stays flat from 16 to 100 edits rather than growing with T. α selects the erasure–preservation operating point.