- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper extends the classification-centered evaluation of approximate domain unlearning to bidirectional image-text retrieval for more reliable forgetting of domain-specific information. To achieve this, the authors propose domain-aware contrastive editing (DACE). By preserving image-text alignment for retaining domains and modifying it for domains to be forgotten through similarity suppression, mismatch reassignment, and rank reversal. The experimental results demonstrate the effectiveness of the proposed method.
Critical strengths
-
Novel problem setting -- This paper addresses the novel and important problem that has not been tackled in previous works. This paper extends the domain unlearning based on classification to bidirectional retrieval.
-
Clear alignment between motivation and method -- This paper is motivated by domain unlearning beyond classification by considering image-text retrieval. The proposed method directly controls the cross-modal alignment, which is effective for both classification and retrieval. These motivation and method align well.
Critical weaknesses
- Limited ablation study -- The proposed method employs the caption-based training for retaining domains, but it is unclear whether the baselines use the same training. Comparing the objectives under otherwise identical training settings would help to identify the source of performance gain.
Questions for authors
- Can authors provide the ablation study that is mentioned in Weakness 1?
5 comments
Response to Reviewer 4FHt (Part 2)
The complete component-wise results on DomainNet, averaged over all possible single forget-domain configurations (=1), are reported below and will be incorporated into the revised manuscript.
| DACE Components | Cls Mem↑ | Cls For↑ | Cls H↑ | T2I Mem↑ | T2I For↑ | T2I H↑ | I2T Mem↑ | I2T For↑ | I2T H↑ | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | ADU | ADU | ADU | 59.96 | 79.14 | 68.11 | 1.53 | 98.95 | 2.99 | 2.04 | 99.17 | 3.99 |
| Retain | DACE | ADU | ADU | 47.09 | 90.57 | 61.94 | 76.74 | 71.66 | 72.42 | 73.60 | 79.13 | 75.16 |
| Forget | ADU | DACE-S | ADU | 61.73 | 75.11 | 67.44 | 36.04 | 89.57 | 49.81 | 38.88 | 86.83 | 51.70 |
| Forget | ADU | DACE-M | ADU | 59.29 | 89.06 | 71.16 | 31.20 | 90.90 | 44.45 | 29.63 | 95.13 | 42.97 |
| Forget | ADU | DACE-H | ADU | 58.15 | 86.03 | 69.38 | 37.47 | 96.10 | 50.90 | 48.21 | 94.72 | 61.67 |
| DDL | ADU | ADU | DACE | 61.47 | 77.41 | 68.25 | 47.54 | 92.58 | 61.52 | 45.81 | 91.89 | 59.55 |
| Retain, Forget | DACE | DACE-S | – | 54.98 | 62.49 | 56.88 | 62.36 | 62.35 | 57.44 | 62.12 | 61.43 | 56.78 |
| Retain, Forget | DACE | DACE-M | – | 54.67 | 90.29 | 68.06 | 58.89 | 85.60 | 68.19 | 60.50 | 89.36 | 70.72 |
| Retain, Forget | DACE | DACE-H | – | 51.57 | 91.54 | 65.86 | 78.84 | 95.63 | 85.20 | 78.57 | 93.44 | 84.48 |
| Retain, Forget, DDL | DACE | DACE-S | DACE | 56.11 | 67.37 | 60.05 | 61.56 | 68.52 | 61.83 | 61.55 | 65.19 | 59.39 |
| Retain, Forget, DDL | DACE | DACE-M | DACE | 51.02 | 92.96 | 65.75 | 54.94 | 87.39 | 65.82 | 52.83 | 93.37 | 65.79 |
| Retain, Forget, DDL | DACE | DACE-H | DACE | 50.03 | 93.38 | 65.01 | 74.88 | 94.23 | 82.11 | 74.94 | 93.59 | 82.34 |
| Retain, Forget, DDL | DACE+CP | DACE-S | DACE | 63.45 | 82.26 | 71.47 | 58.00 | 86.02 | 67.75 | 57.45 | 84.18 | 66.49 |
| Retain, Forget, DDL | DACE+CP | DACE-M | DACE | 61.86 | 90.12 | 73.35 | 56.68 | 87.95 | 67.37 | 54.82 | 90.92 | 66.71 |
| Retain, Forget, DDL | DACE+CP | DACE-H | DACE | 60.05 | 89.41 | 71.84 | 76.30 | 90.56 | 81.86 | 76.99 | 87.87 | 81.34 |
Dear Reviewer,
Thank you again for your time and for reviewing our paper. We have provided detailed responses to your comments and updated the manuscript accordingly. We would appreciate your feedback on whether our responses and revisions have addressed your concerns or if any points require further clarification.
We would be happy to provide any additional details.
Thank you for your time and consideration.
Response to Reviewer 4FHt (Part 1)
We thank the reviewer for the positive assessment of our paper's novelty, soundness, and motivation, particularly the recognition of extending domain unlearning beyond classification to bidirectional retrieval and the alignment between our motivation and proposed method. We also appreciate the constructive feedback.
W1 & Q1: Text supervision and matched comparisons
The source of text supervision depends on the dataset. On DomainNet, paired BLIP-2-generated captions are available and DACE uses the caption associated with each image. On Office-Home and Mini-DomainNet, image-specific captions are unavailable, so DACE uses the standard class-label-derived prompt as the text target. We will make this distinction explicit in the revised manuscript.
DomainNet: All methods use the same images and class labels, but they do not use caption supervision in the same way. DACE and CLIPErase directly optimize image-caption alignment, whereas ADU is classification-oriented: its retain and forgetting losses act on class-prompt predictions and do not optimize instance-level caption alignment. We will therefore avoid suggesting that the methods exploit captions identically and this difference is part of the methodological comparison.
Office-Home and Mini-DomainNet: These datasets contain no image-specific captions. ADU and DACE are trained with the same images, class labels, and class-prompt targets. The classification results in Table 3 of Appendix B therefore provide a matched-supervision comparison that is not confounded by access to image-specific captions.
Scope of the evidence. Office-Home and Mini-DomainNet evaluate classification only. DomainNet is the dataset on which we evaluate both zero-shot classification and bidirectional image-text retrieval. On DomainNet, all methods follow the same downstream evaluation protocol, while their training objectives use the available supervision differently by design. We will revise the paper to distinguish these two conclusions: the additional datasets establish classification performance under matched text supervision, whereas DomainNet provides the evidence for task-spanning domain unlearning.
Additional controlled comparisons with ADU. We also interchanged the retain, forgetting, and DDL objectives of ADU and DACE under identical DomainNet data, training budgets, and forget-domain configurations. Replacing ADU's retain objective, forgetting operator, or DDL with the corresponding DACE component independently increases the bidirectional-retrieval harmonic mean in the reported comparison. The strongest retrieval results arise from combining DACE's contrastive retain objective with DACE-H forget objective. The effect of retain-forget-aware DDL is operator-dependent: it is most useful for the weaker DACE-S suppression objective and is smaller, or can be negative, for DACE-M and DACE-H. Adding the auxiliary class-prompt retain loss recovers classification performance at a modest retrieval cost. These controlled results support the contribution of the alignment-editing objectives without attributing all gains to caption supervision or to DDL.