acceptodds
Under review as a conference paper at ICLR 2027

Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run

Abstract

Privacy auditing aims to empirically assess privacy leakage in machine learning models using membership inference attacks (MIAs), and to derive lower bounds on differential privacy (DP) parameters. Standard privacy auditing inserts a sensitive sample (a canary) into many () randomized training runs, requiring the auditor to detect whether it was included in each run. In one-run auditing, many canaries are included in a single training run, avoiding the high computational cost of multi-run approaches. However, recent theory shows that canaries can interfere with one another, reducing per-canary MIA scores and thereby weakening the audit power. We propose IBIS, a framework that optimizes canaries to maximize their own score while minimizing their impact on others. IBIS combines influence-based greedy initialization with bilevel optimization, introducing a novel regularization term that encourages canaries to be orthogonal in embedding space as a proxy for reducing interference. We propose an efficient bilevel algorithm that incrementally updates a single model at each canary update, avoiding the need to retrain the model from scratch after every update. Experiments show that IBIS offers a better performance-cost trade-off than existing canary crafting approaches, improving MIA metrics while reducing computational cost by up to two orders of magnitude.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy, auditing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ew8uOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IBIS, an efficient canary crafting framework for one-run privacy auditing. The paper addresses the interference among multiple canaries, which can reduce membership inference attack (MIA) performance when canaries are randomly included in a single training run. IBIS combines influence-based greedy selection, which favors high self-influence and low cross-influence, with bilevel optimization that improves canary detectability while encouraging orthogonality in the embedding space. To reduce computational costs, the authors employ approximate implicit differentiation to incrementally update the model and canaries without retraining from scratch at every iteration.

Experiments on CIFAR-10, CIFAR-100, and MNIST demonstrate improved MIA performance over standard canary constructions. Compared with MetaGradient, IBIS achieves better attack performance while reducing canary crafting time by up to two orders of magnitude.

Critical strengths

S1. Meaningful problem formulation and methodological contribution. The paper addresses an important limitation of one-run privacy auditing: interference among canaries. Jointly optimizing canary detectability and non-interference is a meaningful contribution beyond methods that primarily optimize individual canaries. The combination of influence-based initialization and bilevel refinement is well motivated.

S2. Substantial computational improvements with encouraging empirical results. The incremental bilevel optimization avoids repeated model retraining, significantly reducing canary crafting costs. The experiments demonstrate improved MIA performance over existing approaches, and the ablation studies provide evidence that the proposed regularization reduces interference and improves audit performance.

Critical weaknesses

W1. Insufficient theoretical justification for the interference suppression mechanisms. Proposition 4.1 establishes a connection between interference and orthogonality under the metric induced by K−1K^{-1} in least squares regression. However, the proposed regularizer instead enforces ordinary embedding-space orthogonality, which does not necessarily imply low cross-influence in deep neural networks. Furthermore, Equation (1) uses signed cross-influence in the denominator, although negative cross-influence can also cause interference. The objective may also become unstable when the denominator approaches zero. These issues weaken the theoretical justification for the proposed design.

W2. Limited experimental evidence for the broader effectiveness of privacy auditing. The experiments primarily consider relatively small-scale models and datasets. Moreover, the improvement over mislabeled canaries on CIFAR-100 is marginal (0.35 vs. 0.33 [email protected]), while the reported DP auditing results are comparable to MetaGradient rather than substantially better. Since the main motivation is improving privacy auditing, stronger evidence in terms of empirical DP lower bounds and more diverse model architectures would better substantiate the claims. Additionally, part of the runtime comparison relies on previously reported results, making the computational comparison less controlled.

Questions for authors

Q1. Why does Equation (1) use signed cross-influence rather than its absolute value? How are zero or negative denominators handled?

Q2. Why is ordinary embedding orthogonality an appropriate proxy for the K−1K^{-1}-weighted orthogonality derived in Proposition 4.1?

Reviewer_bcUgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes IBIS, a canary-crafting method for one-run privacy auditing that targets interference between canaries. It works in two stages:

  1. Greedy selection. It picks training points whose self-influence is high relative to their maximum cross-influence with canaries already selected.
  2. Bilevel refinement. It optimizes these canaries with an objective comparing a model trained with all canaries against a fixed canary-free reference, plus an embedding-orthogonality regularizer motivated by a least-squares analysis (Prop. 4.1). Only one inner model depends on the canaries, so it is tracked with SOBA-style single-step updates instead of being retrained.

On WRN16-4 (CIFAR10/100) and an MLP (MNIST), IBIS outperforms Random, Mislabeled, and MetaGradient canaries in TPR@5%FPR and AUC, at up to two orders of magnitude lower crafting cost. It performs on par with MetaGradient in DP-SGD auditing.

Critical strengths
  1. Efficiency. Amortized single-step updates replace MetaGradient’s unrolled training over random subsets. Measured crafting time is 0.1h vs. 30h on MNIST. The full one-run pipeline takes about 1.3 GPU hours vs. over 160 for multi-run auditing (App. B.3). This shows that crafted canaries need not cancel the cost advantage of one-run auditing.
  2. Adaptability. The framework incorporates data augmentation and per-example clipping (App. B.1), and aligning crafting with the audited pipeline improves results (Table 3).
Critical weaknesses
  1. The surrogate objective rewards interference. It compares all canaries included vs. all excluded. Similar canaries lower each other’s loss when included together, so unregularized optimization favors interference, which is the opposite of what one-run auditing needs. The regularizer therefore corrects the objective rather than merely “preserving” low interference (Sec. 4.2). This may explain the modest gain at λ=0 (Fig. 1a). The authors should report cross-influence after λ=0 optimization. They should also compare against an objective over a few random subsets or a leave-one-out objective.

  2. The regularizer does not match its motivation. Prop. 4.1 yields K⁻¹-orthogonality under least squares, but the regularizer uses unweighted, unnormalized Euclidean inner products under cross-entropy; If φ is WRN16-4’s penultimate layer (256-d, non-negative after ReLU), 1000 canaries cannot be mutually orthogonal, and a zero inner product requires disjoint supports; The penalty may therefore mainly shrink embedding norms or sparsify activations rather than decorrelate the canaries. The authors should report embedding norms and cosine similarities before and after regularization, and compare against cosine or K⁻¹-weighted variants.

  3. The interference metric does not isolate interference. Normalizing Var_S[Δ(z, S)] by the squared mean gap measures signal-to-noise: a larger mean gap lowers it even if the variance is unchanged. Each Δ involves two training runs, so the variance also includes training randomness. No fixed-S, seed-only noise floor is reported. Fig. 1b thus shows an SNR improvement, not the claimed mechanism. The authors should report unnormalized variance and mean gap separately, together with the noise floor. Ten subsets per canary may also give imprecise estimates.

  4. The comparison with MetaGradient may be unfair. On CIFAR10, the MetaGradient canaries are the authors’ released ones. It is unclear whether they were crafted under the same augmentation + EMA pipeline used for auditing here. IBIS uses the pipeline-aligned variant: Table 2’s 0.31 matches Adapted IBIS in Table 3. Alignment alone raises IBIS from 0.21 to 0.31, so the 0.31 vs. 0.15 gap cannot be attributed to the method itself. The authors should state the crafting setup of the released canaries and compare under the same pipeline, or at least report non-aligned IBIS.

  5. There is no gain in DP auditing. The abstract frames DP lower bounds as a goal of auditing, yet: IBIS and MetaGradient are within one standard deviation at every ε, and MetaGradient has the higher mean at ε∈{2, 4}. At ε≤4, all methods, including Random, are at noise level. The authors acknowledge this in App. B.2. The performance gains are therefore validated only on non-private MIA metrics, and the claims in the abstract and introduction should be qualified accordingly.

  6. Missing baselines, such as greedy selection, greedy selection combined with label flipping, ClipBKD-style canaries, and blank or OOD canaries.

  7. Only small models are evaluated, which does not support the paper’s large-scale motivation.

Questions for authors
  1. What criterion guides the grid search over λ, η, and ρ (App. C.2)? If it is audit performance on audited models, the extra training runs affect the cost comparison and risk overfitting to the evaluation setup.
  2. Table 9 uses ρ=100 without regularization and ρ=1 with it. What is the result for λ=0 at ρ=1?
  3. DP canaries are crafted with a noise-free, clipping-only inner problem. Do the authors attribute the lack of gain under DP to this unmodeled noise?
Reviewer_hYmzOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the problem of strengthening membership inference attacks that are performed in one-run with black box access to the trained model. One-run MIAs start randomly select samples from a canary set to be included or excluded from training, and measure the accuracy of an adversary which guesses membership on the canary set. Such attacks can be quite weak and can be strenghtened by optimizing the canary set. The authors propose a canary crafting method which compared to prior work is ~100 times more efficient and achieves similar or higher rates of adversary accuracy.

To craft the canary set, the authors identify that the canaries can be impacting each other's detectability. They propose maximizing per-canary detectability while minimizing cross-interference by encouraging orthogonality of canaries in the embeddings space. The optimization is performed approximately and iteratively, and requires training 2 additional models on top of the audited one.

Critical strengths
  • Empirical results show clear improvement in efficiency over the prior method and similar or better attack accuracy.
  • The idea of minimizing cross-interference is interesting and utilizing orthogonality as a proxy for cross-interference is intuitive.
  • The paper develops the intuition for their idea very clearly, it is well-written and easy to follow.
Critical weaknesses
  • I would have like to see more comparison to the prior work of Boglioni et al. Boglioni et al use 4 different datasets in their method and train models with 5 different epsilon values providing about 20 datapoints. The paper only shows a subset of the datapoints. Maybe the authors of Boglioni et al would be willing to share their CIFAR100 and MNIST Fashion canaries publicly if contacted?
  • Boglioni et al also mention a method proposed during the review period which they call Gradient Max (https://arxiv.org/pdf/2507.15836, Table 6 and 7), which seems significantly more efficient and can achieve comparable attack accuracy. Were the authors aware of this method and would you be able to comment on its efficiency and maybe compare attack accuracy to Gradient Max?
Questions for authors

Please see above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.