acceptodds
Under review as a conference paper at ICLR 2027

Data-Chain Backdoor: Do You Trust Diffusion Models as Generative Data Supplier?

11 views

Abstract

The increasing use of generative models such as diffusion models for synthetic data augmentation has greatly reduced the cost of data collection and labeling in downstream perception tasks. However, this new data source paradigm may introduce important security concerns. Publicly available generative models are often reused without verification, raising a fundamental question of their safety and trustworthiness. We present Data-Chain Backdoor (DCB), a framework that registers existing backdoor triggers in diffusion models for reproduction in clean-label synthetic data. DCB generates images containing backdoor triggers from ordinary target-class prompts without assuming specific user wording, enabling passive backdoor propagation through synthetic data augmentation. We establish this threat through generator training from scratch and develop an efficient fine-tuning pipeline for trigger registration. Guided by our analysis of trigger registration and reproduction, we further introduce ChainTint, a trigger designed for faithful propagation through DCB and stealth against backdoor defenses. Extensive experiments across diffusion models, downstream classifiers, and defense settings evaluate DCB with existing triggers and ChainTint, including applications to data-scarce learning and dermatological image classification. The DCB framework and ChainTint demonstrate a practical threat to downstream models in generative data supply chains.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
diffusion model, AI security, synthetic data augmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_svbbOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Data-Chain Backdoor (DCB): an attacker fine-tunes the denoiser of a public text-to-image diffusion model so that ordinary target-class prompts produce clean-label images carrying a backdoor trigger — no trigger words, no label flipping, no downstream access. The training objective combines a shared latent-displacement estimate, a "transport supervision" target, supporting denoising losses, and adaptive loss balancing; the authors also design ChainTint, a palette-constrained color-field trigger meant to survive generation. Experiments span five diffusion architectures (CFG-LDM, SD1.5, miniSD, SD3, PixArt-Sigma), five victim classifier architectures, prompt-variation tests, three semantic-consistency filters, five downstream defenses, few-shot learning, and a HAM10000 dermatology case study. On ImageNet-10 with miniSD, ChainTint reaches >=99.6% ASR on all five victims while the synthetic data still improves clean accuracy. Fine-tuning costs 0.5 GPU-hours versus 393 GPU-hours for training a generator from scratch.

Critical strengths
  1. The threat model is realistic and the constraints are actually tested. No downstream access, no textual trigger, and survival of user-side semantic filtering are each evaluated directly. In Table 4, CLIP and LLaVA reject most Eidolon samples while >97% of ChainTint samples pass with ASR intact. If the community's vetting practice is "check semantic consistency and accuracy", this paper shows that practice is not enough.
  2. The evaluation is broad and honest. Five generators, five victims, five defenses, prompt variation, few-shot, and a medical domain, with FID/IS and augmentation utility reported alongside ASR. The authors report where the attack weakens (U-COMBAT and Blended on SD3, PixArt-Sigma) instead of cherry-picking.
Critical weaknesses
  1. Transfer is uneven and the headline results cover a narrow slice. ChainTint gets 99.9% ASR through miniSD but 83.6% through SD3 and 71.1% through PixArt-Sigma; SIG drops to 67.2% on SD3. All main experiments use one target class ("golf ball") on ImageNet-10. The claim of a "practical threat" is therefore well supported mainly for the miniSD/ImageNet-10 configuration, and the paper does not analyze which architectural factor (latent resolution, MMDiT vs U-Net, text encoder) drives the gap.
  2. The defense analysis uses only standard defenses at default settings, and no mitigation direction is explored. An adaptive defender — e.g., trigger inversion applied to the released generator itself, or STRIP with feature-space superimposition — is the natural counterpoint and is absent. Given the conclusion that neither semantic consistency nor accuracy gains certify safety, the obvious follow-up is what would constitute evidence of safety, and the paper leaves that open.
Questions for authors
  1. What explains the ASR gap between miniSD and SD3 for the same trigger (99.9% vs 83.6% for ChainTint)? Is it the MMDiT architecture, resolution, or the VAE? Would the flow-prediction variant (Eq. 9) need a different cmax or kappa?
  2. Was standard data augmentation (random crop/flip) used when training the victims in Tables 2-3? Appendix D says "without training-time augmentation" for the prompt-variation study; please clarify for the main results, since augmentation is known to attenuate clean-label triggers.
  3. Can one released checkpoint carry triggers for several target classes at once, and does per-class ASR degrade? This matters for how damaging a single malicious release can be.
  4. Most configurations use a single run (Appendix E). Could you report mean +/- std over seeds at least for Tables 2-3?
6 comments
AuthorsOpenReview ↗

Author Response to Reviewer svbb (Summary)

We thank the reviewer for the constructive feedback and for recognizing the realistic threat model and the breadth of our evaluation. We summarize our responses below and address each comment in detail in the following sections.

  1. Cross-generator transfer and scope (W1, Q1): We analyzed the transfer gap across miniSD, SD3, and PixArt-Sigma. VAE reconstruction preserves the direction of the ChainTint field (cosine similarity ≈ 0.99), while lower trigger fidelity in generated images coincides with lower downstream ASR.

  2. Stronger defenses (W2): We added generator-side trigger inversion and feature-space STRIP to §5.3. Trigger inversion finds no noise-space signal in DCB generators comparable to a planted backdoor, and feature-space STRIP detects 17% of triggered inputs at a 5% false-positive rate. We also discuss what an effective defense should achieve.

  3. Training-time augmentation (Q2): Tables 2–3 use resizing and random horizontal flips. With added random cropping, color jitter, or RandAugment, ChainTint retains 96.2–98.9% ASR.

  4. Multiple targets in one checkpoint (Q3): One miniSD checkpoint transferred two backdoors to the same classifier, with 99.6% ASR for ChainTint (golf ball) and 91.0% for SIG (tailed frog) (Appendix K).

  5. Random seeds (Q4): Tables 2–3 now report mean ± standard deviation over three seeds. Mean ChainTint ASR across the five classifiers ranges from 99.1% to 99.8%.

AuthorsOpenReview ↗

Author Response to Reviewer svbb (Part 1)

"Transfer is uneven and the headline results cover a narrow slice. ChainTint gets 99.9% ASR through miniSD but 83.6% through SD3 and 71.1% through PixArt-Sigma; SIG drops to 67.2% on SD3. All main experiments use one target class ('golf ball') on ImageNet-10. The claim of a 'practical threat' is therefore well supported mainly for the miniSD/ImageNet-10 configuration, and the paper does not analyze which architectural factor (latent resolution, MMDiT vs U-Net, text encoder) drives the gap."

"What explains the ASR gap between miniSD and SD3 for the same trigger (99.9% vs 83.6% for ChainTint)? Is it the MMDiT architecture, resolution, or the VAE? Would the flow-prediction variant (Eq. 9) need a different cmax or kappa?"

Thank you for raising this interesting question. As discussed in §1, standard fine-tuning alone does not reliably transfer a backdoor through a diffusion model to a downstream classifier. SIG was designed for direct data poisoning, which may help explain its weaker transfer through SD3. Even so, DCB substantially improves downstream ASR over standard fine-tuning for SIG. We designed ChainTint specifically for transfer through generated training data. Although ChainTint’s ASR drops from 99.7% on miniSD to 84.3% on SD3, it still poses a substantial risk to the downstream classifier (revised Table 2). This is particularly concerning given the limited safeguards for this emerging vulnerability. To the best of our knowledge, platform security frameworks such as Hugging Face currently do not perform default checks to detect embedded backdoors like DCB [1]. We chose ImageNet-10 to evaluate DCB on higher-resolution images that better reflect practical use. Appendix B (Table 9) also evaluates DCB on CIFAR-10 with a CFG-DDPM trained from scratch. In that setting, SIG achieved 99.61% downstream ASR at a 5% global poisoning rate.

Where the transfer gap appears. For the generators tested, VAE encoding and decoding preserved the direction of the designed ChainTint field (cosine similarity ≈0.99). Loss of trigger direction during VAE reconstruction is therefore unlikely to explain the ASR drop.

We then measured how faithfully the trigger appeared in generated images. For each generator, we subtracted the mean image generated by its clean checkpoint from that generated by its poisoned checkpoint and measured the cosine similarity of this difference with the designed field.

GeneratorBackboneTrigger fidelity (cosine)Downstream ASR (%)
miniSDU-Net0.92699.7 ± 0.3
SD3MMDiT0.75984.3 ± 1.1
PixArt-SigmaDiT0.71371.1

Across these three generators, lower trigger fidelity in the generated images coincided with lower downstream ASR, suggesting that weaker trigger reproduction during generation may help explain the gap. The miniSD and SD3 ASRs are means ± standard deviations across three seeds (revised Table 2). The PixArt-Sigma ASR is from the original evaluation. For SD3, we kept the same c_max and κ as in the noise-prediction setting without further tuning. Identifying which architectural factor drives the gap is a mechanistic question. Our limited compute resources prevented us from running these controlled comparisons during rebuttal. We appreciate the suggestion and building on the preliminary analysis above, we have updated our conclusion and outlined further controlled experiments in our future work discussion.

References

[1] Huggingface security. https://huggingface.co/docs/hub/en/security

AuthorsOpenReview ↗

Author Response to Reviewer svbb (Part 2)

"The defense analysis uses only standard defenses at default settings, and no mitigation direction is explored. An adaptive defender, e.g., trigger inversion applied to the released generator itself, or STRIP with feature-space superimposition, is the natural counterpoint and is absent. Given the conclusion that neither semantic consistency nor accuracy gains certify safety, the obvious follow-up is what would constitute evidence of safety, and the paper leaves that open."

We added both evaluations to §5.3 of the revised paper.

Trigger inversion applied to the released generator. Motivated by Elijah [1] and TERD [2], we tested whether a defender with access to the released generator could recover a noise-space trigger. We held each generator fixed and optimized an initial-noise perturbation to minimize output variance across a batch of generations. The inversion loss reached 0.004 on a positive control with a planted noise-space backdoor. On the DCB generators, it remained at 0.73–0.78, close to the clean generator’s 0.751. Thus, the inversion test found no comparable noise-space signal in the DCB generators.

Feature-space STRIP. we adapted STRIP to blend features before the classifier head. At 95% pre-screening ASR and a matched 5% false-positive rate, detection improved from 3% with pixel-space STRIP to 17% with feature-space STRIP, leaving most triggered inputs undetected. The figure 5 in §5.3 shows the entropy distributions.

As we show in our response to Q3, DCB can transfer several distinct trigger types, so a defender may not know in advance which trigger to screen for. A promising direction is to study how the diffusion model reproduces these triggers in generated training images and use that understanding to design a DCB-specific defense for the released generator. An effective defense should bring downstream ASR close to the clean-model baseline across triggers while preserving clean accuracy. Developing this defense is beyond the scope of the current paper. We thank the reviewer for the suggestion and have expanded our conclusion and future Work to address this direction.

References

[1] Shengwei An et al. Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift, AAAI, 2024.

[2] Yichuan Mo et al. TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors, ICML, 2024.

Reviewer_Yyu6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents an attack where if a practitioner uses a generative model to generate data for a classifier, the generator can generate injected images that may add back-doors to the classifier outputs. The claim is that the attacker does not need to touch the victim's training pipeline to be successful in that attack.

Critical strengths

I see the following strengths:

1/ The attack is quite realistic. Practioners use generated images from image gen models to train further models. 2/ The evaluations are quite broad that demonstrate the practicality of the attack.

Critical weaknesses

I see the following weaknesses:

1/ Generally, for this attack vector, it is successful only if the attacker's model is adopted. Generally, a mix of datasets and sources are used in model training and there may be some filtering steps. The degree and nature of adoption defines the success of this attack.

2/ Building on above, the impact with different degree of injection during model training is understudied in the paper. Training algorithms may use a mix of data, different starting points (initializations from other models). These considerations need to be studied to make sure that the attack survives.

3/ I'd imagine that this attack will have to generalize to other use cases for image understanding and different schemas for classification. Generalization from that sense is also understudied.

Questions for authors

N/A

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer Yyu6 (Summary)

We thank the reviewer for the constructive feedback and for recognizing the attack’s realism and practicality. We summarize our responses below and address each comment in detail in the following sections.

  1. Checkpoint adoption and data filtering: We clarify our realistic threat model (§1, §3) where users leverage public diffusion models for data augmentation. We demonstrate attack resilience under low poisoning rates and semantic/quality filtering, and provide additional evidence demonstrating real-world applicability and possibility.

  2. Poisoning rate and victim initialization: Appendix I has been added with evaluations across lower injection rates (maintaining 62.1% ASR at 0.14% poison rate) and ImageNet-pretrained victim initializations (achieving 89.8%–91.7% ASR).

  3. Classification with pretrained embeddings and transfer to image retrieval: We added evaluations on pretrained multimodal encoders (Appendix H) and image retrieval (Appendix J), showing successful attack transfer (e.g., 42.4% targeted mAP in retrieval) while preserving clean utility.

AuthorsOpenReview ↗

Author Response to Reviewer Yyu6 (Part 1)

"Generally, for this attack vector, it is successful only if the attacker's model is adopted. Generally, a mix of datasets and sources are used in model training and there may be some filtering steps. The degree and nature of adoption defines the success of this attack."

We agree that DCB relies on a user adopting the released model checkpoint. This requirement is precisely why we formulate a realistic threat model in §3 and highlight the associated non-trivial design challenges in §1. Furthermore, leveraging publicly released third-party models or datasets to train custom architectures is established standard practice. Prior backdoor studies operate under similar adoption assumptions when examining settings where users train on poisoned data sourced from public repositories or external providers [1][2][3].

Practitioners frequently employ publicly released diffusion models to synthesize training data for downstream classifiers, particularly in domains where labeled data are scarce or costly to acquire, such as medical imaging. For instance, Hamidi et al. [4] utilized the public Cheff checkpoint to generate synthetic chest X-rays, augmenting real radiographs to train a disease classifier. Similarly, Korshunov et al. [5] leveraged Arc2Face to synthesize multi-image identities for training AdaFace recognition systems. Crucially, mainstream model repositories such as Hugging Face currently lack automated checks to detect embedded backdoors such as DCB [6].

To directly address the reviewer's point regarding mixed training datasets, we evaluated DCB across varying poisoning rates (Appendix I). Notably, DCB retains a 62% ASR even at a low global poisoning rate of 0.14%. A detailed analysis of poisoning rates and victim model initializations is provided in our response to Weakness 2. Regarding data filtering defenses, Section 5.3 evaluates semantic label consistency checks, and Appendix G extends this to three automated image-quality filters. When calibrated to reject 5% of clean generated images, these filters removed only a minor fraction of ChainTint samples. Following retraining on the filtered datasets, downstream ASR remained above 99% across all semantic filters (§5.3, Table 4) and image-quality filters (Appendix G, Table 13).

References

[1] Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun. “A unified evaluation of textual backdoor learning: Frameworks and benchmarks.” Advances in Neural Information Processing Systems, vol. 35, pp. 5009–5023, 2022.

[2] Rui Ning, Jiang Li, Chunsheng Xin, and Hongyi Wu. “Invisible poison: A blackbox clean label backdoor attack to deep neural networks.” IEEE INFOCOM 2021 – IEEE Conference on Computer Communications, pp. 1–10, 2021.

[3] Giorgio Severi, Jim Meyer, Scott Coull, and Alina Oprea. “Explanation-Guided backdoor poisoning attacks against malware classifiers.” 30th USENIX Security Symposium (USENIX Security 21), pp. 1487–1504, 2021.

[4] Hassan Hamidi, Salamata Konate, Sara Hassani, Andrew Sellergren, Ali Sadeghi-Naini, and Laleh Seyyed-Kalantari. “Clinically Diverse Chest X-ray Synthesis via Cross-Modal Conditioning.” The Second Workshop on GenAI for Health: Potential, Trust, and Policy Compliance.

[5] Pavel Korshunov, Ketan Kotwal, Christophe Ecabert, Vidit Amir Mohammadi, and Sébastien Marcel. “Investigation of accuracy and bias in face recognition trained with synthetic data.” 2025 IEEE International Joint Conference on Biometrics (IJCB).

[6] Huggingface security. https://huggingface.co/docs/hub/en/security

AuthorsOpenReview ↗

Author Response to Reviewer Yyu6 (Part 2)

"Building on above, the impact with different degrees of injection during model training is understudied in the paper. Training algorithms may use a mix of data, different starting points (initializations from other models). These considerations need to be studied to make sure that the attack survives."

To examine whether DCB persists under lower injection rates and different victim initializations, we added two experiments in Appendix I.

Injection rate. We reduced the proportion of poisoned images in the downstream training set from 7.14% down to 0.14%. Even at a minimal 0.14% poisoning rate, DCB achieved a robust 62.1% ASR on ResNet-18.

Poison RateBA (%)ASR (%)
0.14%84.562.1
0.36%84.772.0
0.71%85.379.3
1.79%84.690.5
3.57%84.596.0
7.14%84.399.7

Victim initialization. For each architecture (ResNet-18 and ResNet-50), we compared training from scratch with fine-tuning from clean ImageNet-pretrained weights, using the same downstream training data in both settings. Both pretrained models acquired the backdoor during fine-tuning, reaching 89.8% ASR on ResNet-18 and 91.7% on ResNet-50.

VictimInitializationBA (%)ASR (%)
ResNet-18From scratch84.299.7
ResNet-18ImageNet-pretrained93.489.8
ResNet-50From scratch82.999.8
ResNet-50ImageNet-pretrained93.991.7
AuthorsOpenReview ↗

Author Response to Reviewer Yyu6 (Part 3)

"I'd imagine that this attack will have to generalize to other use cases for image understanding and different schemas for classification. Generalization from that sense is also understudied."

Different schemas for classification. Our original few-shot learning case study demonstrates DCB in settings with limited labeled training data. Appendix H broadens this evaluation across three pretrained multimodal encoders, each fine-tuned with a linear classification head for 1-shot learning. Across all three encoders detailed below, augmenting training with DCB-generated images enhances clean accuracy by 7.3–18.5 percentage points, while maintaining high downstream ASR on real test images ranging from 58.8% to 72.9%.

Image encoderACC, real only (%)ACC++, real + DCB (%)ASR, real + DCB (%)
CLIP-RN5071.189.670.8
CLIP-ViT-B/3284.291.558.8
FashionCLIP80.592.172.9

Other use cases for image understanding. We introduced an image retrieval evaluation, with full experimental details provided in Appendix J. Specifically, we repurposed the clean and DCB-infected classifiers as feature extractors to rank a gallery of clean real images without conducting retrieval-specific fine-tuning. For triggered query samples from non-target classes, the DCB model consistently ranked target-class images significantly higher, achieving a 42.4% targeted mAP compared to 13.4% for the clean baseline. In the absence of the trigger, both models attained approximately 10% targeted mAP and an identical 67.3% clean mAP. These results confirm that the backdoor successfully transfers to image retrieval tasks while preserving clean retrieval utility.

Feature extractorClean mAP (utility)Targeted mAP, no triggerTargeted mAP, trigger
Clean67.310.013.4
DCB67.39.942.4
Reviewer_DyojOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an attack on generative models that produce synthetic data. The approach adds a specific semantic pattern to generated images such that the classifier trained on the data learns that pattern to be associated with this feature. An attacker can later apply the pattern to any other image and get the model to misclassify everything.

Critical strengths

The paper does a good job solidly evaluation proposed approach.

Critical weaknesses

I think the paper is quite clear about the problem that it addresses, yet it is not evident to me that the approach is novel or can be useful for others.

First, I agree that backdooring diffusion models is a nice trick. However, we already know that this is possible and have been done numerous times already.

Second, the attack on color tints looks like will trigger image quality detectors and can be easily filtered. The filters used in 5.3 do not appear to me very strong.

Lastly, the considered classifier on traditional datasets look a bit outdated, i.e. ResNet-18 is unlikely to be used these days for classification, instead people would use a pre-trained multi-modal embedding model and build a linear classifier on top of its embedding. I would add this to the discussion.

Questions for authors

I understand a solid amount of work that were done to get this idea out there, but I doubt that the findings or approaches are particularly novel or applicable to how real pipelines are deployed these days. To the best of my knowledge, a pipeline where a diffusion model generates data for Resnet-18 is not a common design. I am happy to be proven wrong if you could provide some real-world examples given how the embedding models are these days.

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer Dyoj (Summary)

We thank the reviewer for the thoughtful feedback and for recognizing the solidity of our evaluation. We summarize our responses below and address each comment in detail in the following sections.

  1. Novelty and practical usefulness: We clarify the distinct threat model and design challenges (§1, §2). We reemphasize the non-trivial nature of downstream clean-label backdoor transfer and introduce real-world workflow examples.
  2. Image-quality filtering: We have added evaluations with three image-quality filters (LIQE, TOPIQ-NR, MUSIQ) in Appendix G. Downstream ASR remains at 99.6–99.9% after filtering.
  3. Linear classifiers on pretrained multimodal embeddings: We have updated Appendix H with extended evaluations on pretrained image encoders (CLIP-RN50, CLIP-ViT-B/32, FashionCLIP). We indicate consistent improvements in clean accuracy while achieving 58.8–72.9% downstream ASR.
AuthorsOpenReview ↗

Author Response to Reviewer Dyoj (Part 1)

"First, I agree that backdooring diffusion models is a nice trick. However, we already know that this is possible and have been done numerous times already."

"it is not evident to me that the approach is novel or can be useful for others."

Distinct threat model. We apologize for the misunderstanding. We clarify that DCB fundamentally differs from prior diffusion backdooring works in both its underlying threat model and core technical challenges. As detailed in §2 (lines 118–123), whereas conventional backdoors aim to manipulate the output content generated by the diffusion model itself, DCB aims to inject a stealthy, clean-label backdoor into a downstream classifier trained on synthetic images produced by a compromised diffusion model (§1, lines 41–45; Figure 1, "Overview of the DCB attack pipeline"). Furthermore, compromising a diffusion model does not inherently guarantee that the backdoor effect will successfully transfer to downstream classification models.

Design challenges. As we discuss in §1 (lines 67–80), successfully achieving this downstream transfer is non-trivial. The attacker must ensure that the trigger consistently manifests within correctly labeled target-class images across diverse, user-defined prompts. Simultaneously, the trigger pattern must possess sufficient learnability for the classifier to robustly associate it with the target class, even though the attacker maintains no access to or control over the user's downstream training environment (§3, lines 151–161).

Real-world applicability. Standard practices increasingly involve leveraging publicly released diffusion models for data augmentation. For instance, Hamidi et al. trained a disease classifier using a blend of real and Cheff-generated chest X-rays [1], while Korshunov et al. [2] employed Arc2Face to synthesize training data for facial recognition systems. DCB highlights a critical security vulnerability within this realistic workflow: a compromised generator can yield correctly labeled synthetic images that implicitly train a downstream classifier to predict the target class whenever the trigger is present. To the best of our knowledge, the Hugging Face does not document default checks to detect embedded backdoors like DCB [3].

References

[1] Hassan Hamidi, Salamata Konate, Sara Hassani, Andrew Sellergren, Ali Sadeghi-Naini, and Laleh Seyyed-Kalantari. “Clinically Diverse Chest X-ray Synthesis via Cross-Modal Conditioning.” The Second Workshop on GenAI for Health: Potential, Trust, and Policy Compliance.

[2] Pavel Korshunov, Ketan Kotwal, Christophe Ecabert, Vidit Amir Mohammadi, and Sébastien Marcel. “Investigation of accuracy and bias in face recognition trained with synthetic data.” 2025 IEEE International Joint Conference on Biometrics (IJCB).

[3] Huggingface security. https://huggingface.co/docs/hub/en/security

AuthorsOpenReview ↗

Author Response to Reviewer Dyoj (Part 2)

"Second, the attack on color tints looks like will trigger image quality detectors and can be easily filtered. The filters used in 5.3 do not appear to me very strong."

DCB preserves image quality, with clean and backdoored diffusion models achieving comparable generation quality metrics (§5.2, Table 2). To directly address the concern that color-tinted images might be easily identified and filtered by automated image-quality filters, we evaluated three representative models: LIQE [1], TOPIQ-NR [2], and MUSIQ [3]. Each filter was calibrated to reject 5% of an independent reference set of clean generated images, after which it was applied to target-class images from the backdoored miniSD model prior to retraining a ResNet-18 classifier on the filtered dataset.

Quality filterChainTint images removed (%)Downstream ASR (%)
No filter0.099.7
LIQE13.599.6
TOPIQ-NR20.699.6
MUSIQ2.799.9

Notably, even TOPIQ-NR, which removed 20.6% of ChainTint images, reduced the downstream ASR by only 0.1 percentage points compared to the unfiltered baseline. Across all three quality filters, the ASR remained remarkably high at 99.6–99.9%. Comprehensive details have been added to Appendix G.

Furthermore, the semantic integrity checks evaluated in §5.3 align with established practices in diffusion-based data augmentation [4] and pose a formidable barrier to evasion. For instance, when synthetic images were semantically inconsistent with their assigned labels, only 0.5% passed a BLIP-based semantic label validation [5].

References

[1] Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. “Blind image quality assessment via vision-language correspondence: A multitask learning perspective.” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2] Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. “TOPIQ: A top-down approach from semantics to distortions for image quality assessment.” IEEE Transactions on Image Processing, 2024.

[3] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. “MUSIQ: Multi-scale image quality transformer.” 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[4] Eyal Michaeli and Ohad Fried. “Advancing fine-grained classification by structure and subject preserving augmentation.” Advances in Neural Information Processing Systems, 2024.

[5] Mohaiminul Al Nahian, Abeer Matar Almalky, Sabbir Ahmed, Abdullah Al Arafat, Mamshad Nayeem Rizve, and Adnan Siraj Rakin. “Unleashing Stealthy Backdoor Pandemic by Infecting a Single Diffusion Model.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026.

AuthorsOpenReview ↗

Author Response to Reviewer Dyoj (Part 3)

"Lastly, the considered classifier on traditional datasets look a bit outdated, i.e. ResNet-18 is unlikely to be used these days for classification, instead people would use a pre-trained multi-modal embedding model and build a linear classifier on top of its embedding. I would add this to the discussion."

"I understand a solid amount of work that were done to get this idea out there, but I doubt that the findings or approaches are particularly novel or applicable to how real pipelines are deployed these days. To the best of my knowledge, a pipeline where a diffusion model generates data for Resnet-18 is not a common design. I am happy to be proven wrong if you could provide some real-world examples given how the embedding models are these days."

We wish to emphasize that our existing empirical evaluation already extends well beyond traditional architectures like ResNet-18. Indeed, our existing baseline experiments on few-shot learning and medical classification already evaluate classifiers constructed on top of pretrained embeddings (§5.6). To further address the reviewer's suggestion, we have incorporated additional 1-shot classification evaluations across three modern pretrained image encoders. The full experimental configuration and expanded discussions are provided in Appendix H.

Image encoderACC, real only (%)ACC++, real + DCB (%)ASR, real + DCB (%)
CLIP-RN5071.189.670.8
CLIP-ViT-B/3284.291.558.8
FashionCLIP80.592.172.9

Incorporating DCB-generated images consistently improved clean classification accuracy across all three encoders, while maintaining high attack success rates (ASR) on real test images ranging from 58.8% to 72.9%. The FashionCLIP experiment provides a compelling real-world retail case study: trained on Farfetch catalog data for fine-grained product classification, adding DCB-generated images boosted clean accuracy by 11.6 percentage points (from 80.5% to 92.1%) while simultaneously achieving an ASR of 72.9%.

Comments on the paper

4 comments

Thanks for the feedback, as I pointed out in the review, the threat model does not sound practical as people don't train classifiers anymore and the attack pipeline is somewhat weird. The new experiments with CLIP are promising but already dropped attack performance significantly. My main concern remains that a backdoor in diffusion models is a known and well studied problem, the fact that these generated images are used for the next stages seems somewhat a straightforward extension to me. I will keep my score.

AuthorsOpenReview ↗

Author Response to Remaining Concerns (Part 1)

as I pointed out in the review, the threat model does not sound practical as people don't train classifiers anymore and the attack pipeline is somewhat weird.

As discussed in Real-world applicability in Part 1 of our response (Author Response to Reviewer Dyoj (Part 1)), as well as in the Introduction (Section 1), Background and Related Work (Section 2), and Threat Model (Section 3) of the paper, we provide evidence that our data-chain backdoor poses a distinct and realistic security threat, different from prior work on backdoors in diffusion models. Furthermore, both in our original paper submission and in the additional experiments provided in this rebuttal, we have demonstrated that our method remains effective within the context of embedding models.

The new experiments with CLIP are promising but already dropped attack performance significantly.

We would like to further clarify this point:

  1. Comparing training from scratch performance with 1-shot adaptation on pretrained CLIP encoders is not a fair comparison. Prior work shows that backdoor ASR often drops substantially in few-shot learning (FSL) settings [1].

  2. Our 1-shot ASR on CLIP encoders is comparable to attacks specifically designed for CLIP adaptation and few-shot settings. Prior work has shown that conventional backdoor attacks can struggle in few-shot learning [1,2], while BadCLIP achieves 66.4% [2]. Our attack achieves up to 70.8% ASR on CLIP-RN50 and 72.9% on FashionCLIP, while significantly improving clean accuracy on these models.

References

  • [1] Liu, Xinwei, et al. "Does few-shot learning suffer from backdoor attacks?." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 18. 2024.
  • [2] Liang, Siyuan, et al. "Badclip: Dual-embedding guided backdoor attack on multimodal contrastive learning." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
AuthorsOpenReview ↗

Author Response to Remaining Concerns (Part 2)

My main concern remains that a backdoor in diffusion models is a known and well studied problem, the fact that these generated images are used for the next stages seems somewhat a straightforward extension to me.

We emphasize that DCB addresses a fundamentally different security problem from backdoors in diffusion models. As discussed in our paper and 'Author Response to Reviewer Dyoj (Part 1)', DCB has a distinct threat model and poses different design challenges.

To further clarify, DCB is not a straightforward extension of existing diffusion backdoor attacks. It introduces a stronger assumption and distinct design challenges:

  • (i) consistently generating the trigger across diverse prompts without an active attacker during data augmentation. Unlike prior diffusion backdoor attacks that often rely on dedicated trigger tokens, DCB assumes benign downstream users who use ordinary prompts to generate images
  • (ii) constructing the backdoored generator cost-effectively while preserving the trigger details necessary for downstream poisoning. As discussed in Author Response to Reviewer Dyoj (Part 1), backdooring a diffusion model does not necessarily translate into an effective attack on downstream classifiers, which we further demonstrate through experiments below
  • (iii) maintaining generation quality and prompt alignment to ensure that poisoned samples are retained for downstream training.
AuthorsOpenReview ↗

Author Response to Remaining Concerns (Part 3)

My main concern remains that a backdoor in diffusion models is a known and well studied problem, the fact that these generated images are used for the next stages seems somewhat a straightforward extension to me.

Backdooring a diffusion model does not necessarily lead to a successful backdoor attack on downstream classifiers. To test downstream transfer, we evaluate UCE [1], a concept-editing method, and EvilEdit [2], a diffusion backdoor attack, using the same downstream pipeline as DCB. All results are ASR (%).

CNN and transformer victims. Existing diffusion editing methods yield substantially lower downstream ASR than ChainTint.

VictimChainTintEvilEditUCE
ResNet-1899.77.84.3
ResNet-5099.87.24.5
Wide-ResNet-50-299.19.04.2
ViT-B/3299.611.712.8
Swin-T99.719.111.6

Pretrained embedding victims. We further evaluate downstream transfer to classifiers adapted from pretrained image encoders.

Embedding victimChainTintEvilEditUCE
CLIP-RN5070.81.05.3
CLIP-ViT-B/3258.84.06.9
FashionCLIP72.91.74.6

These results demonstrate that editing or backdooring a diffusion generator does not necessarily produce an effective downstream classifier backdoor, highlighting our distinct threat model and design challenge. This observation is also consistent with findings reported in prior work [3].

Given the additional results and clarifications provided above, we would appreciate it if you could let us know whether any concerns remain. If so, we would greatly appreciate any specific supporting evidence or examples.

References

  • [1] Gandikota, Rohit, et al. "Unified concept editing in diffusion models." 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2024.
  • [2] Wang, Hao, et al. "Eviledit: Backdooring text-to-image diffusion models in one second." Proceedings of the 32nd ACM International Conference on Multimedia. 2024.
  • [3] Mohaiminul Al Nahian, Abeer Matar Almalky, Sabbir Ahmed, Abdullah Al Arafat, Mamshad Nayeem Rizve, and Adnan Siraj Rakin. “Unleashing Stealthy Backdoor Pandemic by Infecting a Single Diffusion Model.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.