acceptodds
Withdrawn from ICLR 2027

TrigCon: Certified Defenses for Conformal Prediction against Backdoor Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conformal prediction (CP) is a statistical framework for quantifying the uncertainty of machine learning predictions, which can construct prediction sets with a user-specified probability of containing the true label. However, we observe that CP is vulnerable when the underlying classifier is backdoored, which can substantially reduce the coverage of the true label (e.g., ↓86%) while increasing the attacker-chosen target label's coverage (↑81%). Such vulnerability hinders the deployment of CP in safety-critical applications, yet still remains under-explored. Applying conventional defense approaches to backdoored classifiers poses several challenges: (1) most methods rely on accurately detecting or removing the backdoor, which may fail due to the stealthy and complex nature of backdoor behaviors; (2) these methods often target specific types of triggers and are difficult to generalize to unseen trigger patterns. In this paper, we propose TrigCon, a certified defense against backdoor attacks on CP that can provably resist arbitrary backdoors under bounded trigger sizes. Specifically, TrigCon partitions an input into several disjoint sub-inputs to construct a group of prediction sets, and then aggregates them through voting to produce the final result. This construction bounds the number of prediction sets affected by a bounded trigger, enabling us to establish a marginal coverage guarantee and derive separate trigger-size bounds for retaining the true label and excluding the target label. Extensive experiments on four benchmark datasets show that TrigCon can effectively resist backdoor attacks, achieving a high true-label coverage (e.g., 95% on Tiny-ImageNet) and a low target-label coverage (e.g., 8% on Tiny-ImageNet).

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
conformal prediction, backdoor attacks, certified defenses, machine learning security, adversarial robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5XuYOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a pioneering systematic study on the vulnerability of conformal prediction (CP) to backdoor attacks and introduces TrigCon, a certified defense mechanism. TrigCon partitions an input into T disjoint sub-inputs via a deterministic hash function, constructs prediction sets for each, and aggregates them through label-wise majority voting. Under the constraint of bounded trigger size, this framework provides certified guarantees for both true-label retention and target-label exclusion. However, the method is essentially an instantiation of the deterministic partition-aggregation paradigm (in the lineage of DPA/PatchCleanser/GNNCert) transferred to CP. The certification relies on the restrictive assumption that clean prediction sets are known, the prediction-set-size overhead is substantial, and the study lacks comparisons with existing backdoor defenses.

Critical strengths
  1. The paper formalizes, for the first time, the threat of backdoor attacks against CP. The empirical measurements are striking: BadNet reduces true-label coverage on triggered inputs from 90% to 4% on Tiny-ImageNet (APS) while raising target-label coverage to 97%. The systematic measurement of this attack surface (4 attacks × 4 datasets × multiple hyperparameter configurations) is a valuable contribution in its own right.
  2. Theorem 1 establishes the 1−2α marginal coverage guarantee in the clean setting via Markov's inequality, with a concise and correct proof. Theorems 2 and 3 decompose set-valued certification into a certified tolerance for true-label retention (c*_y) and one for target-label exclusion (c*_yt). This is a sensible generalization of prior single-label certification, and the arguments rely only on the property that each trigger unit affects at most one sub-input—ensured by the disjoint partition—and therefore hold for arbitrary trigger shapes, colors, and locations.
Critical weaknesses
  1. The core technical route—deterministic hash partitioning + disjoint sub-inputs + majority-vote certification against bounded local perturbations—is structurally highly similar to DPA, PatchCleanser, and GNNCert. The differentiation argument (end of Sec. 6) rests mainly on the distinct problem setting and the "set-valued" certification objective, but the methodological increment is limited, and Theorems 2/3 essentially reduce to per-label vote-margin counting.
  2. Theorems 2/3 explicitly assume that "the clean prediction set for each sub-input is known in advance," and c* is computed from clean votes V+/V−. At deployment, the defender may only observe the (possibly) triggered input x̂ and cannot access clean votes. Sec. 5.2(2) acknowledges that triggers are "not explicitly injected" in the certified evaluation. The operational form of the guarantee is thus effectively: "if this input is clean, then adding any trigger of size ≤ c does not change label membership." No procedure is offered for computing or interpreting certification on real test streams, and Table 16 reports only empirical (not certified) metrics on genuinely triggered inputs.
  3. TrigCon uses α = 0.05 to match vanilla CP (α = 0.1) at 90% coverage, so the guarantee is already weaker (1−2α vs. 1−α); meanwhile, prediction sets inflate substantially: on Tiny-ImageNet (200 classes), average set size under APS grows from 16.49 to 42.16 (~2.6×, Table 1), and on CIFAR-100 from 3.60 to 14.15. This cost is dismissed with a single "trade-off" remark, without discussing acceptability in the motivating safety-critical scenarios.
  4. The introduction criticizes empirical defenses (Neural Cleanse, Fine-Pruning, SampDetox, CBD), yet no such defense combined with CP is compared empirically, nor are robust-CP methods (Gendler et al., 2022; Zargarbashi et al., 2024). Comparing only against vanilla CP does not support the claim that the certified route is preferable.
  5. A 5×5 = 25-pixel trigger with T = 16 covers all sub-inputs in the worst case, so certification necessarily fails. Increasing T enlarges the certified radius but degrades sub-input information, requiring sub-image finetuning—yet without finetuning the sets cover nearly all classes (94/100 on CIFAR-100, Table 10), weakening the "model-agnostic" claim (Sec. 3.2, goal 3).
  6. (a) It is unclear whether Table 9's ACC/ASR are measured before or after sub-image finetuning; if finetuning partially removes the backdoor, Table 16's robustness may be partly attributable to finetuning rather than voting. (b) No standard deviations reported. (c) No code/reproducibility statement.
Questions for authors
  1. Given a test input x̂ that may already contain a trigger, how can the defender compute the certified tolerance? Could you provide a symmetric certification based on observed votes V±(x̂) and clarify its relationship to Theorems 2/3?
  2. Are Table 9's ACC/ASR re-measured after sub-image finetuning? How much does finetuning affect ASR?
  3. Why no comparison against "CBD / Neural Cleanse / SampDetox + vanilla CP," ideally including adaptive-attack bypass cases?
  4. For spatially contiguous patches, the number of affected sub-inputs is typically far below the worst-case c; have you considered exploiting contiguity for a tighter certified radius?
  5. Have you tried weighted or confidence-based aggregation instead of the fixed T/2 threshold to reduce set size?
  6. Given knowledge of f and the deterministic partition h, could a dispersed trigger attack TrigCon more effectively under the same budget c? An adaptive-attack experiment would substantiate the worst-case analysis.
Reviewer_6eGtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the vulnerability of conformal prediction to backdoor attacks, which can reduce true-label coverage while increasing target-label coverage. To address this issue, the authors propose TrigCon, which partitions inputs into disjoint sub-images, constructs conformal prediction sets for each sub-image, and aggregates them through majority voting. The paper provides theoretical guarantees for clean-input coverage and certified robustness against bounded-size triggers, and evaluates the method on four image classification datasets.

Critical strengths
  1. The paper addresses a practically relevant problem by investigating how backdoor attacks affect conformal prediction, particularly true-label and target-label coverage.

  2. The paper is generally well organized and clearly written. The proposed method and its theoretical analysis are easy to follow.

Critical weaknesses
  1. The methodological novelty appears limited. TrigCon mainly combines deterministic input partitioning, conformal prediction, and majority voting, which are closely related to existing partition-based certified defenses [1, 2] and the aggregation of conformal prediction sets [3]. Although these works consider different attack settings, the paper does not sufficiently establish what is fundamentally new beyond applying these techniques to backdoor attacks. More direct comparisons with existing certified defenses are needed.

  2. The certified robustness is limited to relatively small perturbation budgets and does not adequately cover the default trigger sizes. According to Theorem 2, the maximum certified number of modified pixels is ⌊(T−1)/2⌋\lfloor(T-1)/2\rfloor, which is only 7 when T=16T=16. However, the default attacks use 3×33\times3 or 5×55\times5 triggers, corresponding to 9 or 25 pixels. While the method achieves high empirical coverage, these results do not establish certified robustness against the full default perturbation budgets.

  3. The choice of TT depends heavily on the expected attack size. Although TrigCon can operate without knowing the trigger size, obtaining a meaningful certificate for a desired attack budget requires an upper bound on the number of modified pixels. Without such prior knowledge, it is unclear how users should select an appropriate TT to ensure sufficient robustness.

  4. Scaling TrigCon to larger triggers requires substantially more partitions, introducing a trade-off between robustness, prediction set size, and computational cost. As TT increases, each sub-image retains less information, leading to larger prediction sets, as shown in Table 1. Moreover, the method relies heavily on fine-tuning with sub-images. Table 10 shows that without fine-tuning, prediction sets can contain nearly all possible labels. Since different values of TT produce different sub-image distributions, it is unclear whether separate fine-tuning is required for each configuration. The effectiveness and practical cost of the method therefore depend strongly on the quality and availability of such fine-tuning.

References

[1] (De)Randomized Smoothing for Certifiable Defense against Patch Attacks, NeurIPS 2020.

[2] Deep Partition Aggregation: Provable Defenses against General Poisoning Attacks, ICLR 2021.

[3] Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning, ICLR 2025.

Questions for authors
  1. What is the main methodological contribution of TrigCon beyond existing partition-based certified defenses [1, 2] and conformal prediction set aggregation [3]? Could the authors clarify which aspects of the proposed method or theoretical guarantees are fundamentally different from these approaches?

  2. Theorem 2 implies that the maximum certified perturbation budget is ⌊(T−1)/2⌋\lfloor(T-1)/2\rfloor, which is only 7 pixels when T=16T=16. However, the default experiments use 3×33\times3 and 5×55\times5 triggers, corresponding to 9 and 25 pixels. Could the authors clarify whether the reported certificates actually cover these default attacks? In particular, how is the effective perturbation budget determined when multiple modified pixels belong to the same partition, and how does upsampling affect this calculation?

  3. How should users select TT when the trigger size is unknown? Is there a principled way to choose TT without assuming a known upper bound on the attack budget?

  4. Table 1 shows that increasing TT substantially increases prediction set sizes, while Table 10 indicates that the method relies heavily on sub-image fine-tuning. Does each choice of TT require separate fine-tuning? How sensitive is the certified performance to the quality of fine-tuning, and what is the additional computational cost as TT increases?

Reviewer_x5eDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows that conformal prediction (CP) on a backdoored classifier loses coverage on triggered inputs: the true label is mostly excluded and the target label mostly included. TrigCon partitions pixels into disjoint groups by hashing and fine-tunes the model on clean sub-images. It then runs CP on each sub-image and keeps the labels that appear in a majority of the sub-sets. Theorem 1 gives 1 - 2α coverage on clean data. Theorems 2 and 3 certify true-label retention and target-label exclusion when the trigger changes a bounded number of pixel positions.

Critical strengths
  • The vulnerability is worth reporting. A standard backdoor almost completely breaks CP coverage while clean behavior is unchanged, and robust CP work has so far focused on evasion and data poisoning.

  • The defense is simple, deterministic and model-agnostic, needs no sampling for certification, and comes with broad ablations.

Critical weaknesses
  • The method and the theory are known. Partition-and-vote certification is well established, from (de)randomized smoothing and DPA to Feature Partition Aggregation (Hammoudeh and Lowd, AAAI 2024). FPA already certifies against ℓ0 evasion and backdoor attacks with disjoint feature partitions. Theorem 1 is Theorem 2.1 of Gasparin and Ramdas (2024) with the same Markov argument, and Theorems 2 and 3 are the standard vote-margin count. Neither prior work is cited.

  • The certificate does not reach the evaluated triggers. With the default T = 16, at most 7 pixel positions can be certified, while the default triggers are 3 × 3 or 5 × 5 patches. The good results on real triggered inputs are therefore empirical. They likely come from the hashing breaking the spatial trigger and from fine-tuning on a large clean set, and no ablation separates these effects from the voting. The guarantee is an ℓ0 bound that does not depend on the backdoor, so it is also vacuous for full-image triggers (blended, warping-based). These are not evaluated.

  • The cost on clean data is large. Prediction sets are two to four times larger than vanilla CP on CIFAR-100 and Tiny-ImageNet. The comparison is also not made at matched coverage, since TrigCon uses a smaller α and over-covers.

  • Vanilla CP is the only baseline. The paper argues that empirical backdoor defenses are unreliable but evaluates none of them combined with CP. It also evaluates no certified alternative, such as a patch-robust classifier wrapped with CP.

Questions for authors

Please answer the questions raised in my review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.