acceptodds
Under review as a conference paper at ICLR 2027

TRACE: TRajectory-Aware Concept Erasure for Flow Matching Models

Abstract

Concept unlearning is essential for suppressing harmful and copyrighted content in text-to-image (T2I) models while preserving their generation utility. While flow matching (FM) models are increasingly widespread, current unlearning techniques are predominantly diffusion-oriented, redirecting model predictions toward safe anchor-conditioned targets or unconditional generations at isolated states. However, directly transferring these diffusion-unlearning objectives to FM models can overlook their inherent continuous flow structure. We theoretically analyze how such adaptation can damage the underlying flow geometry. Flow-aware unlearning is essential for FM models because mismatching the flow paths can introduce conflicting velocity supervision and impose a positive regression floor, which causes a risk of utility degradation during unlearning. To overcome these structural disruptions, we propose TRACE, a flow-aware unlearning framework that erases target concepts via safe flow trajectory replacements. TRACE first jointly selects anchors for different generation seeds using a flow conflict proxy. It then fine-tunes the model along the selected teacher trajectories to replace harmful or undesirable generations while preserving a safe flow structure. Across the eight main-table settings, experiments on Stable Diffusion 3.5 Medium and FLUX.2 4B demonstrate that TRACE outperforms state-of-the-art baselines, reducing target-concept CLIP similarity by up to 7.10%, increasing retained-prompt CLIP alignment by up to 7.78%, and reducing FID by up to 16.55%. Furthermore, under sequential unlearning of 15 concepts on SD 3.5 Medium, TRACE maintains a 74.93% anchor-win rate and 15.29 FID, retaining a clear advantage over the compared baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Flow Matching, Concept Unlearning, Concept Erasure, Flow Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_U4r3OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper presents a new concept erasure method for flow matching models called TRACE, which is supposed to be "trajectory-aware". They postulate that concept erasure can induce conflicting velocities paths in a flow model, which degrades the model utility. It's main novelty seems to lie in a pre-unlearning stage that optimizes the anchor trajectory to reduce the utility-degrading side-effects of the unlearning itself. They evaluate TRACE on many concept-unlearning scenarios and measure erasure efficacy with CLIP-based metrics and utility additionally with FID. TRACE appears to outperform prior work on these metrics, but the appendix comes with extensive additional formal proofs, which are unfortunately not really connected to the main paper and raise some doubts on their true relevance to the main contribution of this work.

Critical strengths
  • Intuitively sensible motivation and overall interesting deep dive into potential pitfalls of concept erasure in these models.
Critical weaknesses
  • Uninformative metrics based on CLIP. CLIP is known to be not a very good signal on its own. Evaluation needs more appropriate measures as established concept classifiers like NudeNet or an additional VLM-based evaluation, especially for identity or nudity unlearning. More qualitative comparisons are needed too. The provided qualitative comparison in Figure 3 does not really reflect a superiority of TRACE over the baselines EA or GEM.
  • Unfortunately, the paper does appear over-formalized (around 23 proofs) which makes it difficult to grasp the essential contribution. Explained pseuocode or a documented code supplement would have helped to better understand the actual method and how it differs to existing methods.
Questions for authors

Clarifications

  • In the paper, it is claimed that achieving effective erasure while preserving utility is particularly challenging for flow matching models (line 52), but you do not provide any empirical evidence for it. Can you provide convincing quantitative support for your claim? This would require analyzing the erasure/utility trade-off for fixed methods across model types.
  • Which prompt benchmarks or prompt templates were used for the evaluation? Can you evaluate on established unsafe prompt benchmarks like T2I-RiskyPrompts, I2P, etc.?
  • Can you explain how Propositions A.16 to A.22 relate to the main paper? In particular, the terms Bayes floor and the first-order scale of mixture ambiguity were puzzling to me.
  • You are writing in the Appendix that the experiments were performed at a resolution of 1024×10241024\times1024 pixels. Is that also the case for the baselines? What hardware was used for these experiments? How many GPUs at once? I am asking because the runtime reported for some of these methods make the impression that the experiments were conducted on a smaller resolution.

Future Work

  • More informative metrics are needed. Anchor-win is an odd choice, because what we primarily care about is whether the erased target concept is truly erased, not whether the surrogate anchor is faithfully generated instead. CLIP is not a very fine-grained signal and needs to be complemented by more qualitative results, proper VLM-based evaluation, and whenever possible by specialized scenario-specific classifiers. Can you extend your main experiments by additional metrics to better compare to related work and give the readers an idea of how much harmful/unwanted generations remain?
  • Can you please provide a direct methodological comparison (perhaps with pseudocode) between TRACE and, for example, ESD or GEM?
Reviewer_PprXOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about concept unlearning specifically customized for flow-matching models. The authors emphasize on the existing diffusion model based methods and how those are suboptimal since they overlook continuous flow structure of FM models. The core idea TRACE is erasing target concepts via replacing them with safe flow trajectories. The authors base their method on 4 proofs and findings (P1, P2, P3, and P4). P1, P2 and P3 contribute to replacing the target concepts and the P4 is about how to select best safe prompts for each seeds to reduce velocity incompatibilities. Authors find safe trajectories using P4 and fine tune the model keeping the rest of generations untouched. The method is tested over two backbones - SD3.5 and Flux2, 15 concepts total. The ablations try to isolate the contributions of P1, P2, P3 and P4.

Critical strengths

Authors identify the needs of customized version of diffusion-model based unlearning for FM models and the design choices are supported by flow matching principles. P1-4 discusses source correspondence, trajectories, velocities and trajectory selection.

The improvements reported in quantitative evaluations are significant. Evaluated across eight concept-backbone settings. For Mickey Mouse, it achieves Anchor-win rates of 86.17% on SD3.5M and 91.95% on FLUX, compared with GEM’s 72.64% and 77.73%

Critical weaknesses

The ablations are incomplete - P1-P3 belongs to improving fine tuning and P4 belongs to selecting complementary trajectories. The way Table 1 ablations organized (native -> p1 -> p1p2 -> p1p2p3 -> TRACE) does not clearly show the contribution of P4. I'd like to see following rows as well - p1p4, p1p2p4. While the design choices and proofs are established in the paper, in this way it'll show the contribution of cleverly selecting trajectories in p4.

Evaluation details are incomplete - Test prompt counts, images per prompt, total number of samples are not clearly mentioned anywhere in the paper. It's not clear whether '5 random seeds' means just 5 image generations or 5 large scale generations starting from 5 random seeds. Also no details about how the prompts were selected for quantitative evaluations are not mentioned. To solidify the results, authors could evaluate on a standard benchmark like I2P or at least provide a justification on why no standard benchmark was not chosen for evaluations.

Missing implementation details - Specially for 'native' and '+p1' rows more implementation details are needed. As an example, for '+p1', were the intermediate states were calculated using forward noising? (I assume this is the case since +p2 uses exact states)

Limited metric choosing - CLIP based metrics famously underperform on detecting smaller features and distortions. While I believe the prompts chosen for evaluations are sufficiently simple for CLIP to handle, MLLM or human evaluation based metric is needed.

Questions for authors

Answer the weakness to the best of your ability.

Reviewer_EQjUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies concept erasure in pretrained flow-matching text-to-image models. It proposes four principles for constructing unlearning supervision: preserving source correspondence, sampling along anchor trajectories, matching instantaneous teacher velocities, and maintaining joint velocity compatibility across the selected trajectories. TRACE first assigns safe anchor trajectories to noise seeds by minimizing a pairwise compatibility score under a context-preservation constraint. It then trains the student under the forget condition to match the selected teacher tangents, together with an objective for preserving non-target behavior. Experiments on SD3.5 Medium and FLUX.2 [klein] Base 4B report improvements in concept removal and utility preservation, including sequential removal of up to 15 concepts.

Critical strengths

A coherent and implementable framework for trajectory-based supervision. Sections 4–5 distinguish several choices that can be conflated in concept editing: source pairing, supervised states, velocity targets, and anchor assignment. Appendix B.1.1 describes the cached pairwise costs and feasible coordinate updates used to solve the assignment problem, making the proposed geometric perspective concrete and testable. Promising reported removal and utility preservation results. In Table 1, TRACE achieves better means than the compared external baselines on all four metrics across all eight main settings. For example, when erasing Mickey Mouse on FLUX, TRACE achieves 91.95% Anchor-win compared with GEM’s 77.73%, while reducing FID from 14.55 to 13.35. Tables 2–3 extend the evaluation to sequential editing on both backbones. Appendix B.1 also explicitly states that evaluation seeds are disjoint from the source seeds used to construct the trajectory bank. Careful qualification of the theoretical results. Appendix A distinguishes teacher-path consistency, regression error within a restricted function class, and semantic performance. Example A.19 shows that nonzero ambiguity in regression targets can coexist with an exact endpoint map. Proposition A.20 explains why reducing the assignment score does not establish an upper bound on optimal regression risk. These qualifications improve the rigor of the theoretical presentation.

Critical weaknesses

The contribution of joint velocity compatibility optimization remains unclear. Section 6.2 compares +P1P2P3 with full TRACE to demonstrate the benefits of joint anchor assignment. However, the assignment procedure in Section 5 involves the choice of replacement concepts, a context-preservation constraint, and velocity compatibility optimization, while the anchor-selection rule used by +P1P2P3 is not explicitly described. The current ablation therefore does not clearly establish whether the gains primarily come from more suitable replacement targets or from reduced velocity conflicts between trajectories. Important experimental settings are insufficiently specified, limiting the comparison of removal and preservation performance. The main empirical claim is that TRACE better preserves non-target generation while erasing the target concept. However, the weight of the retention loss introduced in Section 5 is not specified, nor are the sampling procedures for retention prompts and states or the number of training updates. Appendix B.1 lists the learning rate, batch size, and LoRA configuration, but does not adequately explain how the main baselines are adapted to the two backbones or how their removal and retention parameters are selected. These settings directly affect the trade-off between concept removal and utility preservation. Some descriptions of the results conflict with the tables and require correction. Appendix B.5 on p. 31 states that joint anchor assignment improves CLIP(R) and reduces FID by 1.95 and 2.03 for Monet-style and Simple sketch on FLUX, respectively. However, Table 6 on p. 34 shows that, from +P1P2P3 to TRACE, Monet-style CLIP(R) decreases from 28.64 to 25.92 and FID increases from 13.51 to 15.46. For Simple sketch, CLIP(R) decreases from 28.58 to 25.86 and FID increases from 13.45 to 15.48. The text reverses the direction of the performance changes in these two settings. In addition, Appendix B.2 on p. 29 uses 28.84 and 26.64 to calculate an 8.26% CLIP(R) improvement over EA for Naruto on FLUX. In Table 4 on p. 32, however, the corresponding scores for TRACE, EA, and GEM are 28.55, 25.31, and 27.65. Under B.2’s stated convention of selecting the strongest external baseline, GEM should be the reference, yielding an improvement of approximately 3.25%. These discrepancies concern both the effect of joint assignment and the magnitude of the method’s gains. The tables, calculations, and associated conclusions should be checked against the experimental records.

Questions for authors

What independent benefit does joint velocity compatibility optimization provide? How exactly does +P1P2P3 select anchors? Does it use the same candidate bank, safety screening, and context-preservation constraint as full TRACE? With these conditions and the training configuration held fixed, does TRACE still outperform a method that selects anchors solely based on context preservation? This comparison would help distinguish the benefits of better semantic replacement targets from those of velocity compatibility optimization. How are the removal and retention settings chosen for each method? Please specify TRACE’s retention loss weight, the sampling procedures for retention prompts and states, and the number of training updates. How are the main baselines adapted to the two backbones, and what criteria are used to select their parameters? Do the available results establish that TRACE’s advantage persists when the removal–preservation trade-off is reasonably adjusted? Which results should be used where the text and tables disagree? Appendix B.5 describes the CLIP(R) and FID changes for Monet-style and Simple sketch on FLUX as improvements, whereas Table 6 shows deterioration in both metrics. The 8.26% improvement calculation for Naruto on FLUX in Appendix B.2 also uses values and a baseline that do not match Table 4. Please verify these results and provide corrected explanations. Would the corrections change the conclusions about the benefits of joint anchor assignment or the magnitude of the overall performance improvements?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.