acceptodds
Under review as a conference paper at ICLR 2027

Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models

Abstract

Unified multimodal models (UMMs) integrate image generation and visual understanding, but are existing single-pathway unlearning methods sufficient to suppress target concepts across both? In this work, we study cross-pathway concept unlearning. Our goal is to prevent a model from both generating images of a target concept and identifying it in input images, while preserving other capabilities. To this end, we introduce UUBench, a diagnostic benchmark covering 30 target concepts and 60 semantically related neighboring concepts. Experiments on representative UMMs show that the evaluated single-pathway methods achieve limited joint forgetting: generation-side editors strongly suppress target generation but yield limited forgetting in visual understanding, while understanding-side baselines struggle to achieve substantial forgetting even in their own pathway. Their direct composition also yields limited joint forgetting. We therefore propose UniForget, achieving joint forgetting rates of 80.0% on BAGEL-7B-MoT and 92.7% on Janus-Pro-7B, outperforming representative baselines. Further analysis reveals trade-offs between joint forgetting and capability preservation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, concept unlearning, unified multimodal models, cross-pathway evaluation, datasets and benchmarks

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_R1urOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates cross-pathway concept unlearning in Unified Multimodal Models (UMMs), where target concepts should be suppressed in both image generation and visual understanding while preserving non-target capabilities. The authors introduce UUBench, a diagnostic benchmark for evaluating joint forgetting and capability preservation, and propose UniForget, which combines generation-side weight editing with understanding-side LoRA optimization. Experiments on BAGEL-7B-MoT and Janus-Pro-7B demonstrate the limitations of existing single-pathway approaches and show improvements in joint forgetting.

Critical strengths

Meaningful problem formulation. The paper identifies an important limitation of existing unlearning approaches: forgetting a concept in one pathway does not necessarily lead to forgetting in the other, even within a unified model. This motivates the need for cross-pathway unlearning and provides useful insights into the behavior of UMMs.

Useful benchmark and evaluation protocol. UUBench evaluates forgetting in both generation and understanding, along with neighboring concept retention and capability preservation. In particular, distinguishing Unified Forget from Unified Strict Forget provides a more comprehensive evaluation than measuring target suppression alone.

Critical weaknesses

1. What contributes most to the improvement in joint forgetting? UniForget combines generation-side weight editing with understanding-side LoRA optimization. However, the current comparisons do not fully isolate the benefits of sequential coordination from those of the richer understanding-side training objectives. It would be helpful to compare independent and sequential training under matched supervision to better understand the source of the improvements.

2. Can the authors provide a more fine-grained analysis of cross-pathway forgetting transfer? A central motivation of this work is that forgetting in one pathway does not necessarily transfer to the other. Beyond the aggregate results, it would be interesting to examine how transfer varies across concept categories and model architectures. In particular, analyzing the relationship between architectural parameter sharing and cross-pathway transfer could provide valuable insights into the underlying behavior of UMMs.

3. How does the similarity between target and neighboring concepts affect unlearning? UUBench includes semantically related neighboring concepts to assess unintended forgetting. A more fine-grained analysis based on visual or semantic similarity would be useful to understand when neighboring concepts are most vulnerable to unlearning and how effectively UniForget preserves them.

4. How robust is joint forgetting to indirect concept references? Although the paper evaluates several prompt perturbations and attacks, it would be useful to further investigate scenarios in which the target concept is referenced indirectly rather than explicitly named. For example, can the model still generate or identify a target from its distinctive visual or semantic attributes? Such evaluations would help distinguish robust concept forgetting from suppression of explicit target identities.

Questions for authors

Please refer to the weaknesses.

3 comments

Q1. What contributes most to the improvement in joint forgetting?

Thank you for pointing out these problems! Table 5 examines the understanding-side objectives through cumulative ablations. To isolate the contribution of sequential coordination, we added the matched-supervision comparison.

We hold the generation-side edit, understanding-side supervision bank, LoRA configuration, training objectives, number of training steps, and merge coefficient fixed. In independent composition, we train the understanding-side LoRA on the base checkpoint and merge its update into the generation-edited checkpoint. In sequential coordination (UniForget), we train the same LoRA directly on the generation-edited checkpoint and merge the update using the same coefficient.

ModelCompositionTF ↑DF ↑TU ↑SR ↑Forget ↑S. Forget ↑
BAGELIndependent100.066.774.782.764.043.7
BAGELSequential100.080.072.284.680.047.6
JanusIndependent96.780.077.169.677.343.7
JanusSequential100.092.775.673.292.758.0

With supervision and objectives held fixed, sequential coordination improves Unified Forget by 16.0 percentage points on BAGEL and 15.4 points on Janus. Unified Strict Forget also improves by 3.9 and 14.3 points, respectively, while TU decreases by 2.5 and 1.5 points. The results show that allowing the understanding-side update to adapt directly to the generation-edited checkpoint makes an independent contribution to UniForget’s gains

Q2. How does cross-pathway forgetting transfer vary across concept categories and architectures?

In preliminary experiments, several existing image-to-text unlearning methods achieved limited forgetting on the understanding side of the UMMs we tested. We reported GD and KL as representative baselines. Their low within-pathway forgetting rates leave open whether a more effective understanding-side intervention would transfer to generation.

For the reverse direction, the generation-only edits TIME achieve strong target forgetting, which lets us examine how much forgetting carries over to understanding. We grouped the 30 targets by concept category and measured TF and DF on the generation-only checkpoints.

BackboneCategoryConceptsT2I Forget (TF) ↑I2T Forget (DF) ↑
BAGELCharacter10100.050.0
BAGELLandmark10100.06.0
BAGELObject10100.016.0
BAGELAll30100.024.0
JanusCharacter10100.046.0
JanusLandmark10100.030.0
JanusObject1090.016.0
JanusAll3096.730.7

Characters have the highest DF on both backbones, at 50.0% on BAGEL and 46.0% on Janus. The ordering of landmarks and objects reverses across models. Objects have higher DF on BAGEL (16.0% vs. 6.0%), while landmarks have higher DF on Janus (30.0% vs. 16.0%).

BAGEL shares the edited attention projections across pathways for text tokens, while its generation-side VAE latents use separate projections. Janus uses the same edited backbone projections in both pathways. Even with this sharing, Janus reaches only 30.7% overall DF despite 96.7% TF. Overall, these results suggest that cross-pathway transfer depends on several factors and cannot be explained by concept category or parameter sharing alone.

Q3. How does target–neighbor similarity affect unlearning?

We analyzed UUBENCH’s 60 predefined target–neighbor pairs. We used frozen DINOv2 to encode held-out images and computed cosine similarity between the mean image embeddings of each pair. For semantic similarity, we compared frozen E5 embeddings of fixed, name-free concept descriptions. We computed both scores before inspecting edited outputs and binned each measure separately into low, medium, and high similarity, with 20 pairs per bin.

Nbr Strict counts a paired record as preserved only if the generated image depicts the correct neighbor and meets the image quality criterion, and the understanding answer correctly identifies that same neighbor and meets the answer quality criterion. We swept edit strengths to match TIME + GD and UniForget as closely as possible on target forgetting and computed pair-level Spearman correlations between similarity and preservation loss.

SimilarityBackboneMethodLow ↑Medium ↑High ↑Spearman ρ
VisualBAGELTIME + GD89.088.388.70.04
VisualBAGELUniForget91.090.390.70.02
VisualJanusTIME + GD88.387.788.0−0.03
VisualJanusUniForget90.389.790.00.01
SemanticBAGELTIME + GD98.089.079.00.54
SemanticBAGELUniForget98.791.082.30.45
SemanticJanusTIME + GD98.088.078.00.58
SemanticJanusUniForget98.790.081.30.48

The bin scores are percentages.

At comparable levels of target forgetting, preservation varies little with visual similarity and falls as semantic similarity increases. UniForget improves preservation in every bin, though the most semantically similar neighbors remain the hardest to preserve.

Q4. How robust is joint forgetting to indirect concept references?

In UUBENCH, DF requires a correct generic category, omission of the target name and its aliases, and no refusal. TU separately checks whether non-identity answers are accurate, specific, and informative. These criteria distinguish name suppression from uninformative answers. We added three tests to see whether target identities can still be recovered through indirect references.

  • Numbered choice. For each target image, we randomly order four candidates from the same category, including the target, two close neighbors, and one additional distractor. We ask the model to return only the option number. Option Accuracy is the fraction of trials in which it selects the target. Wrong choices, invalid numbers, and refusals count as incorrect.
  • Multilingual recognition. We use semantically equivalent prompts in Chinese, Spanish, and Japanese. We ask the model to identify the subject of the image in each language and use the standard translated name in generation prompts. We draw four independent samples per condition. GPT-5.6 Sol checks image validity and target recognizability, and checks whether the answer reveals the identity through a name, translation, transliteration, or equivalent expression. ML-S.Forget requires a valid image with no recognizable target and an answer that gives the correct generic category without identity leakage or refusal.
  • Name-free descriptions. We sample three Gemini 3.5 descriptions per target covering appearance, function, and context. We use string matching and other checks to exclude names, aliases, translations, and obvious variants. GPT-5.6 Sol checks target recognizability in images generated from the descriptions alone, without knowing the editing method. For understanding, the model describes the category, visible features, and scene without proper nouns. The judge checks for identity-specific information that cannot be inferred from the image. For an Eiffel Tower image, “a tall iron tower” describes visible features, while “located in Paris” can reveal identity if the location is not visible. We draw four independent samples per condition. Desc-S.Forget requires a valid image with no recognizable target and a correct, informative answer without refusal, target naming, or nonvisual identity clues.
BackboneMethodOption Accuracy ↓ML-S.Forget ↑Desc-S.Forget ↑
BAGELBase95.30.00.0
BAGELTIME + GD72.020.017.8
BAGELUniForget58.731.728.3
JanusBase96.00.00.0
JanusTIME + GD68.025.021.7
JanusUniForget53.338.334.4

All scores are percentages.

UniForget lowers numbered-choice accuracy and raises both strict forgetting scores relative to TIME + GD on both backbones. Together, these results indicate that UniForget makes target identities harder to recover under tests that go beyond English-name matching.

We sincerely appreciate the reviewer’s positive reassessment of our manuscript. Thank you for your thoughtful suggestions and support!

Reviewer_o1uzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies concept unlearning across image generation and visual understanding in unified multimodal models. It introduces UUBench for evaluation and UniForget, which combines weight editing with LoRA updates. Experiments show improved joint forgetting over existing baselines.

Critical strengths

The paper tackles a novel and meaningful question: whether concept unlearning transfers between generation and understanding in unified multimodal models. Its benchmark thoughtfully evaluates joint forgetting alongside utility and neighboring-concept preservation. Experiments on two architectures show promising improvements over the evaluated baselines, while ablations reveal useful trade-offs between forgetting and capability preservation.

Critical weaknesses

My main concern is whether the model actually forgets the concept or simply learns to answer with a generic category instead of its name. Testing on different question templates helps, but is not enough to distinguish these possibilities. More varied tests would be useful, such as selecting an identity by option number, answering in another language, or generating the target from a description without naming it. These tests should also check answer quality, since avoiding the name alone does not demonstrate forgetting.

Questions for authors
  1. Does forgetting hold under more varied tests, such as identity selection by option number, multilingual queries, and generation from descriptions without target names?
  2. Do responses remain accurate and informative under these tests, beyond simply avoiding the target name?
3 comments

W1, Q1 & Q2. Target identity recovery and answer quality

Thank you for suggesting these tests. We added three ways to probe whether the target identity can still be recovered and compared Base, TIME + GD, and UniForget under the same settings.

  • Numbered choice. For each target image, we present four candidates from the same category in random order: the target, two close neighbors, and one additional distractor. The model is asked to return only the option number. Option Accuracy is the fraction of trials in which the model selects the target. Wrong choices, invalid numbers, and refusals count as incorrect.
  • Multilingual recognition. We ask identity questions and request target images in Chinese, Spanish, and Japanese, using standard translated names in the generation prompts. We draw four independent samples per condition. GPT-5.6 Sol checks image validity and target recovery and flags identity leakage through names, translations, transliterations, or equivalent expressions. ML-S.Forget counts a trial as successful if the generated image is valid and does not depict the target, and the understanding answer gives the correct generic category without revealing the identity or refusing to answer.
  • Name-free descriptions. We use Gemini 3.5 to sample three descriptions per target covering appearance, function, and context, and screen them for names, aliases, translations, and obvious variants. For generation, GPT-5.6 Sol checks image validity and target recovery without knowing which editing method was used. For understanding, the model describes the category, visible features, and scene without using proper nouns. The judge checks for identity-specific information beyond what the image shows. For example, “a tall iron tower” describes visible features of the Eiffel Tower, while “located in Paris” can reveal its identity when the location cannot be inferred from the image. We draw four independent samples per condition. Desc-S.Forget counts a trial as successful if the generated image is valid and does not depict the target, and the understanding answer is accurate and informative, gives the correct category, and contains no refusal, target name, or nonvisual clue that reveals the identity.

We manually verified all ground-truth annotations. All scores are percentages.

BackboneMethodOption Accuracy ↓ML-S.Forget ↑Desc-S.Forget ↑
BAGELBase95.30.00.0
BAGELTIME + GD72.020.017.8
BAGELUniForget58.731.728.3
JanusBase96.00.00.0
JanusTIME + GD68.025.021.7
JanusUniForget53.338.334.4

UniForget improves on TIME + GD across all three tests on both backbones. Numbered-choice accuracy drops by 13.3 percentage points on BAGEL and 14.7 points on Janus, showing that the target is harder to recover even when the model only has to select a number. Together, these results show that UniForget reduces the recoverability of target identities beyond English-name suppression.

Recent surveys also identify the complete removal of cross-modal knowledge and resistance to adversarial concept recovery as open problems (Sarwar et al., 2026 and Safavigerdini et al., 2026). In addition, RUB (Xuan and Li, 2026) shows that existing methods can remain vulnerable to attacks that recover residual knowledge even when they pass standard unlearning evaluations. We will therefore clarify in the revised manuscript that UniForget more effectively reduces the recoverability of target identities under the attacks we tested, while fully robust concept unlearning remains challenging.

References

  1. Sarwar et al. (2026). Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks. Findings of ACL 2026.
  2. Safavigerdini et al. (2026). Machine Unlearning in Computer Vision: Survey of Methods from Discriminative to Generative Foundation Models. CVPR Workshops 2026.
  3. Xuan and Li (2026). RUB: Evaluating Residual Knowledge in Unlearned Models. CVPR Workshops 2026.

Thanks for the author's response. All my issues have been resolved and I will maintain my rating.

Thank you for your thoughtful feedback and for taking the time to review our response!

Reviewer_dWDHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates cross-pathway concept unlearning in unified multimodal models and introduces UUBENCH, which pairs generation and understanding tasks for 30 target concepts and 60 neighboring concepts, with disjoint training and evaluation images and templates. They also propose UNIFORGET, combining closed-form generation-side editing with understanding-side low-rank updates and calibrated parameter merging. Experiments on BAGEL-7B-MoT and Janus-Pro-7B demonstrate the utility, scalability, and robustness.

Critical strengths
  1. Compared with evaluating generation or understanding in isolation, UUBENCH checks both pathways in the same edited checkpoint and distinguishes target suppression from image degradation, uninformative responses, and damage to neighboring concepts. This makes the benchmark a useful diagnostic contribution rather than simply a collection of forgetting prompts.
  2. UNIFORGET improves Unified Forget from 31.3% to 80.0% on BAGEL and from 41.3% to 92.7% on Janus relative to TIME + GD, with corresponding gains under the stricter utility-aware metric. Experiments demonstrate the utility, scalability, and robustness.
Critical weaknesses
  1. The understanding-side baselines achieve little forgetting even within their intended pathway, particularly on BAGEL, where GD and KL obtain only 0.7% and 0.0% DF. Their low joint-forgetting rates therefore cannot distinguish a failure of cross-pathway transfer from ineffective understanding-side adaptation. Besides, UNIFORGET uses a method-specific supervision bank and richer objectives, making the improvement over TIME + GD difficult to attribute specifically to pathway coordination.
  2. Despite strong unified forget rates, unified strict forget reaches only 47.6% on BAGEL and 58.0% on Janus. Target visual utility declines from approximately 96% in the base models to 72.2% and 75.6%, while neighboring generation specificity reaches only 84.6% and 73.2%. General generation performance also decreases and multi-concept editing produces severe additional degradation. Therefore, UNIFORGET demonstrates effective suppression, but its utility-preserving performance remains constrained, especially when forgetting multiple concepts.
Questions for authors
  1. UUBENCH defines understanding-side forgetting primarily through target-name and alias suppression, while the utility supervision of UNIFORGET retains most of the original answer after removing these names. How does the benchmark distinguish identity suppression from name avoidance when a name-free description still uniquely identifies the target
  2. BAGEL uses modality-specific transformer parameters, whereas Janus uses a shared transformer backbone. Which edited parameters participate in both pathways in each model? How does such a distinction affect the interpretation that shared parameters do not imply shared forgetting?
  3. Section 6.3 evaluates generation-side and understanding-side scaling separately. Does the composition introduce additional interference affecting unified strict forget?
5 comments

W1. Does understanding-side forgetting transfer to generation? & What accounts for UniForget's gains?

Does understanding-side forgetting transfer to generation?

We thank the reviewer for pointing out that the scope of our conclusion was unclear. In our original experiments, GD and KL achieved limited forgetting within the understanding pathway and yielded no joint forgetting. Several other understanding-side objectives also showed limited within-pathway forgetting in preliminary experiments, so we reported GD and KL as representative baselines. These results left open whether an effective understanding-side intervention would transfer to generation. We will clarify this limitation earlier in the revised manuscript.

To test transfer when understanding-side forgetting is substantial, we added an Understanding Only experiment that applies only UniForget's understanding-side LoRA update. On BAGEL and Janus, this intervention achieves Direct Forget (DF) rates of 96.0% and 96.7%, respectively, while the generation-side Target Forget (TF) rates are 3.3% and 0.0%. This indicates limited transfer to generation for this intervention on the two evaluated models.

What accounts for UniForget's gains?

We agree that the comparison with TIME + GD measures the gain of the full UniForget method, since the two methods differ in generation-side editing, understanding-side supervision, and training objectives. Table 5 examines the understanding-side objectives through cumulative ablations. To isolate the contribution of sequential coordination, we added a matched-supervision comparison.

We hold the generation-side edit, understanding-side supervision bank, LoRA configuration, training objectives, training steps, and merge coefficient fixed. In Independent composition, we train the understanding-side LoRA on the base checkpoint and merge its update into the generation-edited checkpoint. In Sequential coordination (UniForget), we train that LoRA on the generation-edited checkpoint and merge its update using the same coefficient.

All scores are percentages.

ModelSettingTF ↑DF ↑TU ↑SR ↑Forget ↑S. Forget ↑
BAGELUnderstanding Only3.396.064.996.23.31.3
BAGELIndependent100.066.774.782.764.043.7
BAGELSequential (UniForget)100.080.072.284.680.047.6
JanusUnderstanding Only0.096.772.091.20.00.0
JanusIndependent96.780.077.169.677.343.7
JanusSequential (UniForget)100.092.775.673.292.758.0

Relative to independent composition, sequential coordination raises joint forgetting from 64.0% to 80.0% on BAGEL and from 77.3% to 92.7% on Janus. Strict joint forgetting improves by 3.9 and 14.3 percentage points, respectively, while target-image description utility (TU) decreases by 2.5 and 1.5 points. With supervision and training objectives held fixed, these gains show that training the understanding-side update on the generation-edited checkpoint improves joint forgetting on both models.

W2. The trade-off between concept forgetting and utility preservation

We thank the reviewer for pointing out this limitation. UniForget substantially suppresses target concepts, but preserving model utility while achieving reliable forgetting remains a challenge for UMM unlearning. To quantify this trade-off, UUBench reports Unified Strict Forget alongside Unified Forget. Unified Strict Forget requires a single paired record to satisfy all of the following conditions: the generated image omits the target concept while maintaining image quality, the identity answer does not identify the target, and the non-identity answer omits the target name while remaining visually accurate and informative. Simply avoiding the target name or giving generic, uninformative answers is therefore insufficient to satisfy the metric.

Our results show that stronger target forgetting can come at the cost of model utility. UniForget provides two hyperparameters for tuning this trade-off. On the understanding side, Section 6.2 and Figure 3 show that increasing the LoRA merge coefficient ρ from 0.4 to 0.6 on BAGEL raises Direct Forget (DF) from 80.0% to 93.3%, while Target Utility (TU) falls from 72.2% to 55.6%. On the generation side, Table 6 in Appendix C.3 shows that increasing the edit strength α from 0.75 to 1.0 on Janus raises Target Forget (TF) from 90.0% to 100.0%, while Strict Neighbor Retention (SR) falls from 78.6% to 73.2%.

In the revised manuscript, we will discuss these results in the Limitations section. UUBench evaluates both forgetting and utility preservation, enabling future work to compare methods on both dimensions.

Q1. Can the target identity be recovered without matching its English name?

In UUBENCH, Direct Forget (DF) requires a correct generic category, no target name or alias, and no refusal or evasion. Target Utility (TU) separately checks whether non-identity answers are visually accurate, sufficiently specific, and free of major hallucinations. These criteria guard against uninformative answers.

Besides, we added three tests of identity recovery:

  • Numbered choice. For each target image, we randomly order four same-category candidates consisting of the target, two close neighbors, and one additional distractor. The model is instructed to return only the option number. Option Accuracy is the fraction of trials where it selects the target. Wrong choices, invalid numbers, and refusals count as incorrect.
  • Multilingual recognition. We ask identity questions and request target images in Chinese, Spanish, and Japanese, using standard translated names in the generation prompts. We draw four independent samples per condition. GPT-5.6 Sol checks image validity and target recovery and flags identity leakage through names, translations, transliterations, or equivalent expressions. ML-S.Forget requires a valid image without the target and an understanding answer that gives the correct generic category without identity leakage or refusal.
  • Name-free descriptions. We sample three Gemini 3.5 descriptions per target covering appearance, function, and context, and screen them for names, aliases, translations, and obvious variants. For generation, GPT-5.6 Sol checks image validity and target recovery without knowing the editing method. For understanding, the model describes the category, visible features, and scene without proper nouns. We check for identity-specific information beyond what the image shows. For an Eiffel Tower image, “a tall iron tower” describes visible features, while “located in Paris” may reveal identity if the location is not visible. We draw four independent samples per condition. Desc-S.Forget requires a valid image without the target and a correct, informative, non-refusal answer without target naming or nonvisual identity clues.

We manually verified all ground-truth annotations. All scores are percentages.

BackboneMethodOption Accuracy ↓ML-S.Forget ↑Desc-S.Forget ↑
BAGELBase95.30.00.0
BAGELTIME + GD72.020.017.8
BAGELUniForget58.731.728.3
JanusBase96.00.00.0
JanusTIME + GD68.025.021.7
JanusUniForget53.338.334.4

Relative to TIME + GD, UniForget lowers numbered-choice accuracy and improves both strict forgetting metrics on both backbones. These results indicate reduced target-identity recoverability across the tested conditions.

Q2. Parameter sharing across the two pathways

We thank the reviewer for prompting us to clarify how the two models share parameters. BAGEL uses partial parameter sharing based on token type. Specifically, the default self-attention projections we edit (k_proj and v_proj) process both text and ViT image tokens in the understanding pathway, as well as text-conditioning tokens in the generation pathway. The VAE latent tokens in the generation pathway use separate generation-specific projections (k_proj_moe_gen and v_proj_moe_gen), which our method leaves unchanged. Thus, the edited projections are shared across pathways for a subset of token types.

Janus uses distinct visual input and output modules, with a single Transformer backbone shared by both pathways. The same edited backbone projections (k_proj and v_proj) are used during both image understanding and image generation.

We selected these models because they represent two common UMM designs and allow us to test whether our findings hold across different patterns of parameter sharing. BAGEL shows that cross-pathway forgetting remains limited with partial sharing. Janus shows that even when both pathways use the same edited backbone projections, strong forgetting in one pathway does not automatically produce comparable forgetting in the other. Across the two architectures we tested, parameter sharing alone was insufficient to ensure cross-pathway transfer of behavioral forgetting.

Q3. Measuring joint forgetting under multi-concept editing

We extend the original single-concept definition to evaluate one checkpoint jointly edited on a set of NN target concepts, denoted by CNC_N. Before inspecting the edited outputs, we fix the aligned evaluation manifest EcE_c for each target concept cc. We then evaluate the jointly edited checkpoint on the disjoint union of these manifests.

ECN=⨄c∈CNEc.E_{C_N} = \biguplus_{c \in C_N} E_c.

We apply the original joint criteria to this union. The angle brackets denote the mean over all eligible paired records in the union, with the predicates multiplied within each record.

Forget(N)=100⟨gd⟩.\mathrm{Forget}(N) = 100 \langle gd \rangle.

S.Forget(N)=100⟨sdvτ⟩.\mathrm{S.Forget}(N) = 100 \langle sdv\tau \rangle.

For N>1N > 1, the unit of evaluation is the paired record. Each score is the percentage of eligible records across all edited targets that meet the corresponding joint criterion on the same checkpoint. For N=1N=1, this definition reduces to the original single-concept definition. The first row of each table reports the corresponding result from Table 2. All scores are percentages.

BAGEL

# conceptsForget ↑S. Forget ↑
180.047.6
568.940.0
1059.343.1
2051.634.1

Janus

# conceptsForget ↑S. Forget ↑
192.758.0
568.053.3
1054.942.2
2043.129.8

The results show that joint forgetting remains possible as more concepts are edited, although both joint metrics are lower at 20 concepts than at one. The joint criteria require both pathways to succeed on the same record, so failures in either pathway can reduce the overall score. Unified Strict Forget declines as the number of edited concepts increases overall, while generation-side target suppression remains high in the separate scaling experiment in Figure 4. We will report the joint scaling results in the revised manuscript and explicitly discuss utility preservation under multi-concept editing as a limitation.

We hope future methods can achieve more reliable concept forgetting while better preserving the models' original capabilities.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

We sincerely thank you, the SPC, and all reviewers for the time and effort devoted to evaluating our submission and for the constructive feedback throughout the review process.

This work provides the first systematic study of cross-pathway concept unlearning in unified multimodal models (UMMs), revealing that forgetting in image generation does not reliably transfer to visual understanding, or vice versa, even when the two pathways share model parameters. We make two main contributions: UUBench, a unified diagnostic benchmark that jointly evaluates cross-pathway forgetting, model utility, and neighboring-concept preservation within the same edited checkpoint; and UniForget, a coordinated unlearning method that substantially improves joint forgetting across two representative UMM architectures. Beyond improving performance, our analyses characterize how cross-pathway transfer depends on concept category and architecture, and expose key trade-offs involving identity recoverability, utility preservation, and multi-concept scalability. The reviewers were positive overall, with one clear accept and two weak accepts. Among them, Reviewer R1ur recognized the significance of the problem and the value of our benchmark, while Reviewer o1uz confirmed that all concerns were resolved after rebuttal.

Our rebuttal and revised manuscript address the following concerns.

  1. Identity forgetting beyond name suppression. In response to Reviewers o1uz and R1ur, we added numbered-choice, multilingual, and name-free description tests that do not rely on English target names and also assess output validity and answer quality. UniForget consistently outperforms TIME + GD on both backbones. For example, numbered-choice accuracy decreases from 72.0% to 58.7% on BAGEL and from 68.0% to 53.3% on Janus, while strict forgetting improves under multilingual and name-free evaluations. These results show reduced identity recoverability beyond surface-level name avoidance.

  2. Contribution of sequential coordination. In response to Reviewers dWDH and R1ur, we compared independent composition and sequential coordination while holding the generation edit, supervision, LoRA configuration, objectives, training steps, and merge coefficient fixed. Sequential coordination improves Unified Forget by 16.0 points on BAGEL and 15.4 points on Janus, and Unified Strict Forget by 3.9 and 14.3 points, respectively. This isolates the contribution of UniForget’s sequential design.

  3. Cross-pathway transfer and architecture. In response to Reviewer dWDH, we added an understanding-only experiment. It achieves 96.0–96.7% understanding-side forgetting but only 0.0–3.3% generation-side forgetting. Conversely, generation-only editing achieves 96.7–100% generation forgetting but only 24.0–30.7% understanding forgetting. Category-level analysis and architectural clarification further show that shared parameters alone do not guarantee behavioral transfer across pathways.

  4. Neighbor preservation and scalability. In response to Reviewer R1ur, we analyzed the 60 target–neighbor pairs using DINOv2 visual similarity and E5 semantic similarity. Semantically closer neighbors are more vulnerable, but UniForget improves preservation over TIME + GD in every similarity bin. In response to Reviewer dWDH, we also evaluated joint editing of 1, 5, 10, and 20 concepts. Joint forgetting remains feasible, although performance declines as more concepts are edited.

  5. Forgetting–utility trade-off. We clarified that Unified Strict Forget requires generation and understanding to jointly satisfy forgetting, validity, and utility criteria. Ablations on the LoRA merge coefficient and generation edit strength further quantify the trade-off between stronger forgetting and capability preservation. We explicitly acknowledge that fully eliminating residual knowledge, preserving utility, and scaling to many concepts remain open challenges.

All new experiments and discussions have been incorporated into the revised manuscript and highlighted in blue. We believe they directly address the reviewers’ concerns regarding identity recoverability, method attribution, cross-pathway transfer, neighboring-concept preservation, and scalability.

We respectfully hope that these results and clarifications, together with the reviewers’ positive assessments, will receive further consideration in the final evaluation. Thank you again for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.