- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes IDOff, a privacy-protection method for tuning-free personalization, designed to achieve both effective identity protection and high-quality image generation. Unlike personalization methods such as DreamBooth or LoRA, this approach focuses on models that can extract and replicate a person’s identity from reference images without any fine-tuning.
Critical strengths
-
The paper considers the inherent trade-off between identity protection and generation quality, a key challenge in existing privacy-protection methods. This is a highly relevant and practical research objective.
-
The introduction of "Identity-Agnostic Anchors" provides a clear optimization objective for perturbing identity features, rather than simply pushing them away from the original representation.
Critical weaknesses
- The paper characterizes existing methods, such as Anti-DreamBooth and SimAC, as "adversarial attacks." However, the primary intent of these techniques is to safeguard user identity privacy, effectively making them proactive privacy-defense mechanisms. While it is technically accurate to describe them as "adversarial attacks" against generative models, the paper should more clearly distinguish between the technical perspective of attacking a generative model and the user-centric perspective of privacy protection. Furthermore, while the paper’s title highlights "Stealthy Adversarial Attack," the actual research objective is to protect user identity rather than to carry out malicious attacks. This phrasing may lead to misunderstandings regarding the paper’s research focus and its threat model.
- This work focuses specifically on optimizing "Identity Carriers" within the context of tuning-free personalization. However, fine-tuning-based personalization methods learn subject identities by updating model parameters. It remains unclear whether the perturbations generated by IDOff can effectively prevent such models from learning and reproducing the original identity.
- In real-world applications, images uploaded or shared by users are often subject to operations such as JPEG compression, scaling, or cropping. Furthermore, before being used for personalization, such images might be processed by an adversary using techniques like "diffusion-based purification." These operations could weaken or even completely eliminate the adversarial perturbations. Given that practical identity privacy protection must account for various image-processing workflows, robustness experiments are essential to demonstrate the method's real-world utility.
- Although the paper conducts transfer experiments across five personalization models not used during the optimization process, it reports only Identity Similarity, omitting results for generation quality and prompt consistency.
- IDOff requires joint gradient optimization across multiple personalized generative models, involving multiple iterations to produce protected images. In contrast, IDProtector can generate protective perturbations in a single forward pass using a pre-trained perturbation generator. It remains unclear whether the gains in generation quality provided by IDOff are sufficient to offset the additional computational costs.
- The experiments lacks independent ablation studies for each core component. For instance, the specific performance contributions of different types of anchors, Residual Regularization, and gradient coordination strategies remain unquantified. Furthermore, the paper lacks sensitivity analysis for key hyperparameters.
- The paper employs four text prompts during the optimization phase, but it is not explicitly stated whether these prompts are strictly distinct from those used in the final evaluation. If the same or highly similar text conditions are used for both optimization and testing, the reported results may not accurately reflect the method’s ability to generalize to unseen prompts. In real-world applications, unauthorized users could employ arbitrary text prompts to generate portraits in various poses, settings, or styles.
Questions for authors
- Can IDOff effectively prevent the reconstruction of the original identity even when the protected image undergoes JPEG compression, scaling, cropping, or purification processes based on diffusion models?
- Does the paper ensure that the identities and text prompts used for perturbation optimization are strictly separated from those used in the final evaluation?
7 comments
Full per-metric robustness to diffusion-based purification (DiffPure, GrIDPure, Noisy Upscaling)
5. Full per-metric transfer across five unseen generators (W4).
Direct finding: Identity suppression and prompt consistency transfer reliably to all five unseen generators, while facial and whole-image quality vary by architecture and metric.
Reusing the same saved protected photographs without adaptation, the table below reports every individual metric--ArcFace, AdaFace, FaceScore, DSL-FIQA, whole-image MUSIQ, CLIP-T, and SCRFD FDFR--across all five unseen generators and all four methods (50 owners x 4 scenes = 200 scheduled generations per method; lower ArcFace, AdaFace, FDFR better; higher quality/CLIP-T better):
| Generator | Method | ArcFace | AdaFace | FaceScore | DSL-FIQA | MUSIQ | CLIP-T | FDFR |
|---|---|---|---|---|---|---|---|---|
| ConsistentID | Untreated | 0.5427 | 0.4986 | 4.2483 | 0.8545 | 76.04 | 0.2790 | 0.025 |
| ConsistentID | IDProtector | 0.2954 | 0.2609 | 3.5194 | 0.8417 | 74.76 | 0.2720 | 0.050 |
| ConsistentID | IDGuardian | 0.3450 | 0.3053 | 3.6133 | 0.8470 | 75.53 | 0.2780 | 0.050 |
| ConsistentID | IDOff | -0.0555 | -0.0284 | 3.7502 | 0.8356 | 74.58 | 0.2777 | 0.050 |
| FastComposer | Untreated | 0.2975 | 0.2721 | 4.3865 | 0.6592 | 73.75 | 0.2593 | 0.000 |
| FastComposer | IDProtector | 0.1632 | 0.1480 | 3.8452 | 0.6289 | 71.39 | 0.2574 | 0.030 |
| FastComposer | IDGuardian | 0.1745 | 0.1582 | 4.1758 | 0.6153 | 71.90 | 0.2589 | 0.010 |
| FastComposer | IDOff | 0.1329 | 0.1251 | 4.1356 | 0.6385 | 72.06 | 0.2611 | 0.010 |
| PhotoMaker v2 | Untreated | 0.5121 | 0.4799 | 4.9678 | 0.8604 | 74.36 | 0.2969 | 0.115 |
| PhotoMaker v2 | IDProtector | 0.2818 | 0.2446 | 4.7655 | 0.8552 | 74.72 | 0.2951 | 0.120 |
| PhotoMaker v2 | IDGuardian | 0.3788 | 0.3419 | 4.8180 | 0.8310 | 73.12 | 0.3021 | 0.115 |
| PhotoMaker v2 | IDOff | -0.0652 | -0.0301 | 4.7923 | 0.8669 | 75.27 | 0.3082 | 0.120 |
| UniPortrait | Untreated | 0.5907 | 0.5588 | 5.4735 | 0.8129 | 76.36 | 0.3086 | 0.035 |
| UniPortrait | IDProtector | 0.3807 | 0.3480 | 5.3720 | 0.8141 | 75.95 | 0.3100 | 0.055 |
| UniPortrait | IDGuardian | 0.4627 | 0.4259 | 5.4841 | 0.8026 | 76.02 | 0.3149 | 0.060 |
| UniPortrait | IDOff | -0.0702 | -0.0348 | 5.4019 | 0.8254 | 76.45 | 0.3136 | 0.047 |
| PuLID | Untreated | 0.6238 | 0.5764 | 5.1937 | 0.7779 | 71.02 | 0.3155 | 0.110 |
| PuLID | IDProtector | 0.2700 | 0.2368 | 4.9486 | 0.7531 | 68.70 | 0.3106 | 0.095 |
| PuLID | IDGuardian | 0.4618 | 0.4125 | 5.1431 | 0.7487 | 69.76 | 0.3168 | 0.115 |
| PuLID | IDOff | -0.3529 | -0.2792 | 5.3190 | 0.7654 | 69.79 | 0.3191 | 0.115 |
Paired 95% confidence intervals (IDOff minus untreated) and valid pair counts (out of 200):
| Unseen generator | ArcFace diff CI | AdaFace diff CI | FaceScore diff CI | DSL-FIQA diff CI | MUSIQ diff CI | CLIP-T diff CI | Valid (ID/Face/Img) |
|---|---|---|---|---|---|---|---|
| ConsistentID | [-0.6275, -0.5645] | [-0.5501, -0.4988] | [-0.6766, -0.3218] | [-0.0302, -0.0094] | [-1.95, -0.97] | [-0.0053, 0.0027] | 189/200/200 |
| FastComposer | [-0.1804, -0.1472] | [-0.1611, -0.1315] | [-0.3240, -0.1869] | [-0.0313, -0.0121] | [-2.65, -0.94] | [-0.0006, 0.0041] | 198/198/200 |
| PhotoMaker v2 | [-0.6081, -0.5458] | [-0.5354, -0.4841] | [-0.3345, -0.0273] | [-0.0004, 0.0138] | [0.42, 1.41] | [0.0082, 0.0147] | 176/200/200 |
| UniPortrait | [-0.6921, -0.6215] | [-0.6212, -0.5593] | [-0.1542, 0.0151] | [0.0050, 0.0201] | [-0.18, 0.36] | [0.0024, 0.0079] | 188/199/199 |
| PuLID | [-1.0172, -0.9325] | [-0.8920, -0.8151] | [0.0209, 0.2261] | [-0.0177, -0.0072] | [-1.58, -0.86] | [0.0018, 0.0054] | 174/200/200 |
IDOff achieves strictly lower ArcFace and AdaFace similarity than untreated references and both official baselines on all five unseen generators (all 10 identity CIs strictly negative). CLIP-T is preserved or significantly improved on all five models; FaceScore significantly rises on PuLID, matches untreated on UniPortrait, and declines on ConsistentID, FastComposer, and PhotoMaker v2. Against official baselines at the matched 8/255 budget (removing our earlier unofficial PGD row and retaining IDGuardian's 224x224 input pipeline), IDOff achieves higher mean FaceScore than released IDProtector on all five generators (strictly positive CIs on ConsistentID, FastComposer, and PuLID) and strictly higher CLIP-T on all five. Against official IDGuardian, IDOff achieves strictly higher FaceScore on PuLID (CI [0.0731, 0.2795]), higher mean FaceScore on ConsistentID (CI [-0.0320, 0.3144]), statistically indistinguishable FaceScore on FastComposer, PhotoMaker v2, and UniPortrait (all CIs include zero), and strictly higher DSL-FIQA on four of five generators.
Full per-metric transfer evaluation and paired 95% CIs across five unseen generators
5. Full per-metric transfer across five unseen generators (W4).
Reusing the same saved protected photographs without adaptation, the table below reports every individual metric--ArcFace, AdaFace, FaceScore, DSL-FIQA, whole-image MUSIQ, CLIP-T, and SCRFD FDFR--across all five unseen generators and all four methods (50 owners x 4 scenes = 200 scheduled generations per method; lower ArcFace, AdaFace, FDFR better; higher FaceScore, DSL-FIQA, MUSIQ, CLIP-T better):
| Generator | Method | ArcFace | AdaFace | FaceScore | DSL-FIQA | MUSIQ | CLIP-T | FDFR |
|---|---|---|---|---|---|---|---|---|
| ConsistentID | Untreated | 0.5427 | 0.4986 | 4.2483 | 0.8545 | 76.04 | 0.2790 | 0.025 |
| ConsistentID | IDProtector | 0.2954 | 0.2609 | 3.5194 | 0.8417 | 74.76 | 0.2720 | 0.050 |
| ConsistentID | IDGuardian | 0.3450 | 0.3053 | 3.6133 | 0.8470 | 75.53 | 0.2780 | 0.050 |
| ConsistentID | IDOff | -0.0555 | -0.0284 | 3.7502 | 0.8356 | 74.58 | 0.2777 | 0.050 |
| FastComposer | Untreated | 0.2975 | 0.2721 | 4.3865 | 0.6592 | 73.75 | 0.2593 | 0.000 |
| FastComposer | IDProtector | 0.1632 | 0.1480 | 3.8452 | 0.6289 | 71.39 | 0.2574 | 0.030 |
| FastComposer | IDGuardian | 0.1745 | 0.1582 | 4.1758 | 0.6153 | 71.90 | 0.2589 | 0.010 |
| FastComposer | IDOff | 0.1329 | 0.1251 | 4.1356 | 0.6385 | 72.06 | 0.2611 | 0.010 |
| PhotoMaker v2 | Untreated | 0.5121 | 0.4799 | 4.9678 | 0.8604 | 74.36 | 0.2969 | 0.115 |
| PhotoMaker v2 | IDProtector | 0.2818 | 0.2446 | 4.7655 | 0.8552 | 74.72 | 0.2951 | 0.120 |
| PhotoMaker v2 | IDGuardian | 0.3788 | 0.3419 | 4.8180 | 0.8310 | 73.12 | 0.3021 | 0.115 |
| PhotoMaker v2 | IDOff | -0.0652 | -0.0301 | 4.7923 | 0.8669 | 75.27 | 0.3082 | 0.120 |
| UniPortrait | Untreated | 0.5907 | 0.5588 | 5.4735 | 0.8129 | 76.36 | 0.3086 | 0.035 |
| UniPortrait | IDProtector | 0.3807 | 0.3480 | 5.3720 | 0.8141 | 75.95 | 0.3100 | 0.055 |
| UniPortrait | IDGuardian | 0.4627 | 0.4259 | 5.4841 | 0.8026 | 76.02 | 0.3149 | 0.060 |
| UniPortrait | IDOff | -0.0702 | -0.0348 | 5.4019 | 0.8254 | 76.45 | 0.3136 | 0.047 |
| PuLID | Untreated | 0.6238 | 0.5764 | 5.1937 | 0.7779 | 71.02 | 0.3155 | 0.110 |
| PuLID | IDProtector | 0.2700 | 0.2368 | 4.9486 | 0.7531 | 68.70 | 0.3106 | 0.095 |
| PuLID | IDGuardian | 0.4618 | 0.4125 | 5.1431 | 0.7487 | 69.76 | 0.3168 | 0.115 |
| PuLID | IDOff | -0.3529 | -0.2792 | 5.3190 | 0.7654 | 69.79 | 0.3191 | 0.115 |
Paired 95% confidence intervals (IDOff minus untreated) and valid pair counts (out of 200):
| Unseen generator | ArcFace diff CI | AdaFace diff CI | FaceScore diff CI | DSL-FIQA diff CI | MUSIQ diff CI | CLIP-T diff CI | Valid (ID/Face/Img) |
|---|---|---|---|---|---|---|---|
| ConsistentID | [-0.6275, -0.5645] | [-0.5501, -0.4988] | [-0.6766, -0.3218] | [-0.0302, -0.0094] | [-1.95, -0.97] | [-0.0053, 0.0027] | 189/200/200 |
| FastComposer | [-0.1804, -0.1472] | [-0.1611, -0.1315] | [-0.3240, -0.1869] | [-0.0313, -0.0121] | [-2.65, -0.94] | [-0.0006, 0.0041] | 198/198/200 |
| PhotoMaker v2 | [-0.6081, -0.5458] | [-0.5354, -0.4841] | [-0.3345, -0.0273] | [-0.0004, 0.0138] | [0.42, 1.41] | [0.0082, 0.0147] | 176/200/200 |
| UniPortrait | [-0.6921, -0.6215] | [-0.6212, -0.5593] | [-0.1542, 0.0151] | [0.0050, 0.0201] | [-0.18, 0.36] | [0.0024, 0.0079] | 188/199/199 |
| PuLID | [-1.0172, -0.9325] | [-0.8920, -0.8151] | [0.0209, 0.2261] | [-0.0177, -0.0072] | [-1.58, -0.86] | [0.0018, 0.0054] | 174/200/200 |
IDOff achieves strictly lower ArcFace and AdaFace similarity than untreated references and both official baselines on all five unseen generators (all 10 identity-difference CIs strictly negative). CLIP-T is preserved or improved on all five models; FaceScore rises on PuLID, matches untreated on UniPortrait, and declines on ConsistentID, FastComposer, and PhotoMaker v2. Against baselines at the matched 8/255 budget (removing our earlier unofficial PGD row, and retaining IDGuardian's 224x224 input pipeline), IDOff outperforms released IDProtector on FaceScore across all five generators (and MUSIQ on four of five), outperforms IDGuardian on FaceScore on ConsistentID and PuLID, and outperforms IDGuardian on DSL-FIQA on four of five.
Privacy framing, fine-tuning scope, and full per-metric robustness to JPEG, downsampling, and cropping
Thank you for highlighting how the title and threat model read from the photograph owner's perspective.
Why the revised setup changed. Rather than patching issues piecemeal, we updated our protocol once and re-ran all evaluations uniformly (explaining numerical differences from September):
Clean-224isolating IDGuardian's mandatory 224x224 downscaling).1. Privacy objective and terminology (W1).
We agree and will revise the title, abstract, and Section 1 to state the photograph owner's proactive privacy goal explicitly: adding bounded perturbations before sharing photos so unauthorized tuning-free models fail to reproduce their identity while still generating plausible, prompt-aligned portraits. We will distinguish the technical mechanism (adversarial optimization) from the defensive threat model, and describe Anti-DreamBooth, SimAC, IDProtector, and IDGuardian as proactive privacy defenses.
2. Scope relative to fine-tuning (W2).
IDOff acts on inference-time identity carriers, whereas DreamBooth and LoRA update model parameters. We will state this boundary clearly and note that fine-tuning protection remains a separate open question.
3. Full per-metric robustness to JPEG compression, downsampling, and cropping (W3 / Q1).
Direct finding: Identity protection survives all tested transforms across all three main generators on both ArcFace and AdaFace, while generated quality stays close to or above equally processed untreated controls.
Protected and untreated references underwent identical processing before generation: JPEG Q85, bicubic downsampling to 75% of original width and height (178x218 -> 134x164), and a seeded random crop retaining 80-100% of each dimension independently, followed by bicubic resizing to the original size (identical boxes across both arms). Cells give paired IDOff / equally processed untreated values (200 scheduled pairs, 50 owners; 199-200 valid pairs everywhere except InstantID Crop with 170 ID pairs across 45 owners and 196 quality pairs across 49 owners):
IDOff keeps both ArcFace and AdaFace substantially below equally processed untreated controls across all 12 pairs (JPEG Q85 weakens protection most), while quality and CLIP-T stay close to or above untreated controls. During optimization we adapted IDProtector's affine augmentation (Song et al., Sec. 4.5; six shared Gaussian offsets, std = 0.003).