acceptodds
Under review as a conference paper at ICLR 2027

Improving Certified Few-Shot Adaptation without Changing Clean Predictions

Abstract

Few-shot predictors can be misled by correctly labeled copies of their support examples. Copying changes class counts and can reverse predictions even when class-conditional feature distributions stay fixed. We separate count-dependent coefficients from support statistics and hold these coefficients at their initial values. This normalization preserves every clean prediction for trimmed-distance and exponential-cache scores, with certificates for their complete modified scores. For the cache, we prove that normalization cannot reduce robustness to a single cross-class stored-source copy when all sources are mutable. The remaining copy sensitivity depends on within-class kernel deviations. Across eight text and image encoders, normalization reduces copy harm without changing any clean prediction. On two image tasks, normalized Tip-Adapter retains 87.5% clean accuracy, and 85.4% of predictions remain correct under every single cross-class stored-source copy, compared with 80.4% for zero-shot classification. Under the broader threat of four arbitrary bounded-feature replacements, certified accuracy rises from zero to 28.3%. Fixing initial coefficients can thus preserve adaptation gains while improving resistance to support-set poisoning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Certified Robustness, Data Poisoning, Few-Shot Adaptation, Count Normalizationa

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EYucOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies support-set poisoning in frozen-feature few-shot classifiers when source replacements change class counts. It separates count-dependent coefficients from support statistics and fixes those coefficients at their pre-attack values. The correction is applied to trimmed-distance scores and Tip-Adapter caches. The authors establish clean-prediction preservation, derive certificates for bounded-feature replacements and exact margins for single-copy attacks, and evaluate the modified classifiers on text and image classification tasks.

Critical strengths

The correction is inexpensive and preserves the original clean predictions exactly. The paper also evaluates its consequences carefully through certificates, exhaustive single-copy enumeration, and shared-cache attacks with held-out queries. These results substantiate that the modification can reduce the measured vulnerability.

Critical weaknesses
  1. The central methodological contribution is too elementary to provide sufficient technical novelty. The essential operation is to express a class sum as the current class count multiplied by a class mean, then replace the current count with its pre-attack value. In Equation (8), this amounts to applying the classwise factor n0,c/ncn_{0,c}/n_c. The trimmed-distance specialization similarly freezes the initial retained-count coefficient. Clean-score preservation follows directly because the initial and current counts coincide before an attack. In my view, this coefficient substitution does not constitute a sufficiently substantial new adaptation or robustness method. The general coefficient-freezing formulation largely restates the same operation.

  2. The assumed attack setting has limited general applicability. The strongest empirical results concern copying already stored, correctly labeled support records within a frozen cache. Although the paper also provides certificates for broader bounded-feature replacements, those guarantees remain conditional on fixed feature computation and a trustworthy initial count snapshot. The practical relevance of this interface is insufficiently established, particularly for pipelines in which contamination occurs during support-set construction or influences preprocessing or model adaptation. The experiments therefore demonstrate robustness within a specific interface without establishing how broadly the improvement transfers to realistic poisoning scenarios.

Questions for authors
  1. What is the principal methodological or theoretical contribution beyond freezing the pre-attack count-dependent coefficients?

  2. What concrete deployment scenario motivates the assumed attacker access and trustworthy initial count snapshot? How would the proposed correction apply when contamination occurs before that snapshot is established?

4 comments

Thank you for the detailed review, and for noting that the correction is inexpensive, preserves clean predictions exactly, and is evaluated carefully. Changes in the revised PDF are in blue.

Q1. What is the contribution beyond freezing the count-dependent coefficients?

We agree that the operation is elementary and that clean preservation follows at once. That is intended, since such a correction can be added to an existing predictor at no cost. We also do not claim the operation as new. It has the form of prior-ratio weights (Lipton et al., 2018), which the revision now cites. What we claim are three results, none of which follows from clean preservation.

(1) Normalization never loses robustness at general budgets (new Theorem 2). Take an all-mutable cache and any budget mm below the initial class sizes. Every prediction that the original keeps under all mm copies, or under all mm arbitrary bounded-feature replacements, is also kept after normalization, and the theorem gives the original's exact worst case. This is not automatic. The same operation loses robustness once some sources are protected (exact counterexample, minimum gap 1/70 → −9/770), and it fails for some other count coefficients (App. W). A variant that normalizes only the mutable part restores inclusion with protected sources (App. O.2), and App. W gives the conditions for general coefficients.

(2) A ceiling for every clean-preserving count correction (new Section 5.6, Table 3). No rule that keeps the original predictions whenever the counts equal the reference counts can beat the original's robustness under count-preserving attacks, and this includes enforcing fixed counts (Prop. 8). On our cache panel, normalization meets this ceiling under four arbitrary replacements (226 of 226, out of 800 pairs) and comes within four predictions of it under four copies (619 vs. 623). Reaching the ceiling is an empirical finding on this panel, not a theorem. It tells us that the remaining failures come from content, and fixing those needs a different predictor.

(3) A poisoning route outside existing few-shot certificates. FCert and LeFCert certify threats that keep the class counts fixed (L100–103). Correct-label copies act through the counts alone (Fig. 1). In native Tip-Adapter, single copies flip 325 of 700 correct predictions, and normalization reduces this to 17.

Q2. Deployment scenario, the count snapshot, and contamination before the snapshot

The scenario is now described in a new paragraph of Section 3.1. A task owner fixes the class vocabulary, the encoder and the kk-shot design, and a separate collector or a writable record store supplies the records. SuS-X, for example, builds labeled caches from external URLs whose content can change after indexing (Carlini et al., 2024). The attacker controls some records but not the configuration. We do not claim an attack on a deployed system.

No snapshot is needed. The reference count is part of the design, n0,c=kn_{0,c}=k in a kk-shot task. If kk is not stored, the reference is the received total divided by the number of classes, which replacement leaves unchanged (App. Q.1). On 5,104 received inputs the scores match those obtained with known kk. Unknown class proportions cannot be identified from counts alone, and we state this as a limit.

Contamination before any cache exists is the same replacement, because records are encoded independently. Poisoned raw manifests encoded from scratch give identical features, scores and predictions (App. N.5).

The harder case is contamination that reaches preprocessing or adaptation (Section 6, "Beyond frozen features"). If records are weighted by n0,c/ncn_{0,c}/n_c before any support-dependent computation, the class-weighted empirical measure does not change when counts change, so deterministic shared preprocessing and fixed-step training are preserved (App. O.1). We tested this with actual refitting and training:

  • With nonlinear preprocessing and ridge refitted after every single copy (400 source-disjoint queries), robust-correct predictions rise from 335 uncorrected to 341 with normalization and 349 with duplicate removal, against 312 for zero-shot. The gain is modest, and on CIFAR-FS the learner trails zero-shot (App. P.4).
  • In full CLIP visual-encoder training (87.8M parameters, AdamW) under whole-class replication, sampling by the reference weights restores the clean trajectory, with 760 of 858 correct against 730 uncorrected (App. U).

Poisoning that changes content remains outside these guarantees, as Section 6 states.

Presentation. We added a contributions paragraph, moved the key results into the main text, and added a reviewer guide before Appendix A.

We hope this answers your concerns and that you will reconsider your assessment. If there is a specific correctness problem behind the soundness score, please tell us and we will address it.

Dear Reviewer EYuc,

The discussion period closes on November 18 (AoE), and we would value your view on whether our response addresses your two questions. On the contribution, we agree that the operation is elementary. What the revision puts forward are results about it. Theorem 2 shows that normalization never loses robustness at general budgets when all sources are mutable. The same statement fails once some sources are protected, so it does not follow from the substitution itself. Prop. 8 (Section 5.6) bounds what any clean-preserving count correction can achieve. On deployment, no trusted count snapshot is needed, because the reference count is the shot count fixed by the task design (Section 3.1, App. Q.1).

If the soundness score reflects a specific proof or experiment that you consider incorrect, please point us to it and we will respond right away. Thank you again for your time.

We thank the authors for their response and the additional theoretical results, which address part of concerns. However, it remains unclear how far the robustness results extend to settings where changes to support-set content affect preprocessing or training.

Thank you for reading our response and for the follow-up. The remaining question is a fair one, and we ran a new experiment to answer it. The guarantees extend exactly as far as class counts do. Past that point they do not hold, and no correction based on counts can make them hold. We measured what happens there. The updated PDF reports it in a new Appendix Z, and Section 6 states the outcome.

The guarantee covers every change that alters the class counts but leaves the records of each class as they were, for example when a whole class is repeated. For such changes, a pipeline that applies the reference weights before every support-dependent step has the same deterministic preprocessing and the same fixed-step training trajectory as the clean run (Prop. 7, App. O.1).

It stops when a replacement brings new content into the support set, because the training data itself has then changed. Replace one record by a different record of the same class. Every count stays the same, so a correction that looks at counts has nothing to act on, yet the shared statistics move. We now bound this movement. In our five-way ten-shot design, the weighted support distribution moves by at most 0.038 in total variation after one replacement and 0.153 after four (new Prop. 15). This limits how far a shared mean can move. It is too coarse to certify the predictions of a refitted learner, and we do not claim such a certificate.

We therefore ran the experiment that your question describes. Stored records were replaced by images that are not in the support set, and after every replacement the shared preprocessing was recomputed and the adapter refitted. The two learners are those of App. O.5: ridge after centering, and ridge after centering and unit normalization, which is a nonlinear shared step. For one replacement the enumeration is complete, over every stored position and every candidate image. For two to four replacements we report attacks found by a beam search, so those numbers are upper bounds on robustness. The table gives robust-correct predictions out of 400 source-disjoint queries when the inserted image carries its correct label. Zero-shot keeps 312.

learnerclean1 replacement: uncorrected / normalized / ceiling2: uncorrected / normalized4: uncorrected / normalized
ridge355302 / 328 / 332209 / 29080 / 198
nonlinear preprocessing372319 / 325 / 335262 / 282145 / 196

The ceiling is the most that any count correction can reach (Prop. 8). We read three things from the table.

First, normalization still helps when preprocessing and the adapter are refitted on new content, and it never hurt. With one replacement, where the enumeration is complete, no prediction that survived without it was lost with it. The search found no such case at larger budgets either.

Second, what remains is mostly content. For the nonlinear learner, replacements that change no count already take the 372 clean-correct predictions down to 335. Of the 16 predictions that a count correction could still recover, normalization recovers 6. For ridge it recovers 26 of 30.

Third, the range is short. With one new image the refitted learners keep 325 and 328 predictions against 312 for zero-shot. This difference is not statistically resolved, and it comes from EuroSAT alone. From two replacements on, both learners fall below zero-shot. If the attacker may also choose the label, the nonlinear learner is below zero-shot already at one replacement (263). The 800-pair cache panel with two encoders shows the same pattern (Table 43).

So the direct answer is this. When several records with new content can reach preprocessing or training, we have no guarantee beyond removing the count effect. Normalization still improves on the uncorrected learner there, but zero-shot or a larger support set is the safer choice. For full visual-encoder training we have only the trajectory-level evidence of App. P.5, where a content-changing replacement leaves a nonzero parameter difference under every correction. The paper's claims concern the count effect, and the revision now states this boundary with numbers.

We hope this settles the remaining point. If it does, we would be grateful if you would consider updating your assessment. We are glad to answer anything further before the discussion closes.

Reviewer_W8FgOpenReview ↗
Rating
3(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows that replacing support records changes class counts and can flip few-shot predictions through count-dependent coefficients. The authors fix these coefficients at their values on the initial support set. Without changing any clean prediction, the method provides certificates for trimmed-distance and exponential-cache scores and reduces copy-poisoning harm on text and image tasks.

Critical strengths

This paper clearly characterizes how support set replacement changes few-shot decisions through class counts and count-dependent coefficients, and it removes this effect with a simple correction that fixes the initial coefficients. The correction provably preserves every clean prediction, and the paper also provides certificates for the complete score and an exact single-copy margin theorem for the cache. The experiments use a prespecified confirmation design on fresh samples and consistently reduce copy harm across eight text and image encoders. The results are reported honestly, and the reproduction materials are complete.

Critical weaknesses

The original rule in this paper is the authors’ own extension of LeFCert to variable class counts. LeFCert itself is defined for a C-way K-shot support set with a fixed trimming parameter, so part of the corrected vulnerability comes from this extension. The authors could argue whether this setting is common in practical systems. The exact single-copy theorem and the nondegradation result hold only when all sources are mutable and only for cross-class stored-source copies. The authors themselves give a counterexample with protected sources, and no corresponding result exists for multiple replacements, so the scope of the theoretical contribution is narrow. The authors could discuss conditions under which it generalizes.

Questions for authors

Please refer to weaknesses.

4 comments

Thank you for the careful and encouraging review. Your comments on scope are the reason the main text now has a theorem for multiple replacements (Theorem 2), and they made us state more precisely where each guarantee holds. Changes are in blue in the revised PDF.

1. The original rule is our own variable-count extension of LeFCert

Yes, that is right. The submission says so at L159–161, and we do not attribute the vulnerability to native LeFCert, which fixes both the shot count and the trim MM. We added two clarifications to Section 3.2.

The first is that the effect does not come from recomputing the trim. If MM is fixed in advance, at the largest symmetric trim that is valid at every reachable class size, the retained-sum coefficient nc−2Mn_c-2M still changes with the counts. In this fixed-MM control, normalization reduces single-copy harm from 113 to 32 of 960 text pairs and from 45 to 12 of 300 image pairs (App. N.4).

The second is that Tip-Adapter needs no extension at all. Its classwise kernel sum depends on the counts as published, and it shows the largest effects in the paper. Single-copy harm goes from 325 to 17 among 700 correct predictions, and certified predictions from 0 to 226 of 800.

2. Is this setting common in practice?

We added a paragraph, "Where count changes arise", to Section 3.1. The setting we have in mind is one where the task owner fixes the vocabulary, the encoder and the kk-shot design, and a separate collector or a writable record store supplies the records. SuS-X is an example. It builds a labeled support cache from images at external URLs, and the content behind a URL can change after indexing (Carlini et al., 2024). Replacing a record with one from another class changes the class counts and leaves the total unchanged. The reference count is part of the task design, either kk or the received total divided by the number of classes (App. Q.1). Since records are encoded independently, contamination at collection time is the same replacement, which we checked by encoding poisoned raw manifests from scratch (App. N.5). We have not observed such an attack on a deployed system, and the paper does not claim one.

3. Multiple replacements

The new Theorem 2 covers this (Section 3.6, proof in App. N.1). Assume all sources are mutable and m<min⁡cn0,cm<\min_c n_{0,c}. The attacker may make any mm same- or cross-class copies of stored sources, with donors reusable, or mm arbitrary replacements of unit features and labels. The original cache's exact worst winner–rival gap is then gwc−α(Tw,m+mHc)g_{wc}-\alpha(T_{w,m}+mH_c), where Tw,mT_{w,m} is the sum of the mm largest winner kernels and HcH_c is the largest rival-donor kernel (or 1 for arbitrary features). The worst gap of the normalized cache is at least this value, so every prediction that is robust under the original stays robust. In the experiments (Table 3), the original keeps 375, 176, 24 and 0 of 800 predictions under one to four copies, and normalization keeps 682, 659, 645 and 619.

4. Protected sources, and conditions under which the result generalizes

The counterexample remains valid for full-class normalization, and we keep it in the paper. The attacker cannot delete protected winner evidence, but normalization can dilute it. The minimum gap is 1/70 for the original and −9/770 after normalization. Normalizing only the mutable part of each class removes the problem: S^c=bc+α(Pc+McMc′Vc′),\widehat S_c=b_c+\alpha\Big(P_c+\frac{M_c}{M_c'}V_c'\Big), where PcP_c is the protected kernel sum and Vc′V_c' is the current mutable sum over Mc′M_c' records (McM_c initially). This keeps the clean scores and gives robust-set inclusion whenever m<min⁡cMcm<\min_c M_c, including when protected donors are copied into mutable positions. On the counterexample the minimum gap becomes 3/10 (Section 3.6, App. O.2). For full-class normalization, it is enough that the winner's mm largest kernels are mutable (App. N.2).

On generalization more broadly, the revision gives three further results, each with its conditions stated. For another count coefficient ac(n)a_c(n), inclusion holds if ac(n0−m)≤ac(n0)≤ac(n0+m)a_c(n_0-m)\le a_c(n_0)\le a_c(n_0+m). For bounded positive kernels an exact finite LP criterion decides the question, and a(n)=1/100+1/na(n)=1/100+1/n is a coefficient for which inclusion fails (App. W). If the initial supports are distinct and duplicates are removed, copying reduces exactly to deletion. This gives an exact certificate and raises four-copy robust-correct predictions from 619 to 660 of 800 (App. P). Finally, if records are weighted before any support-dependent preprocessing or training, those shared computations are unchanged under pure count changes (App. O.1, U).

Thank you again. We would be happy to discuss any of these points further.

Dear Reviewer W8Fg,

Thank you again for the encouraging review. As the discussion period closes on November 18 (AoE), we wanted to ask whether the revision answers your comments on scope. Two changes came directly from them. Theorem 2 extends the inclusion result from one cross-class copy to multiple copies and arbitrary replacements, and the mutable-component variant restores inclusion when some sources are protected (Section 3.6, App. O.2). If either falls short of what you had in mind, we would be happy to clarify.

Reviewer_W8FgOpenReview ↗

Adjust my acore

After reading the other reviewers' comments, I identified some additional weaknesses in the paper. Therefore, I have decided to lower my score to 3 to better align my evaluation with those of the other reviewers.

Thank you for letting us know about the change. We would like to respond to it while the discussion is still open, and for that we need to know which weaknesses you have in mind. Your comment says that you identified additional weaknesses after reading the other reviews, but it does not name them. Could you tell us which ones they are?

In case the change reflects the concerns raised in the other two reviews, here is where the revision answers them.

Reviewer MFdY asked why enforcing a fixed number of supports per class does not already solve the problem, and why the certificate under four arbitrary replacements (28.3%) is below zero-shot. The new Section 5.6 answers both. Any correction that keeps the original predictions at the reference counts, enforcement included, is bounded by a ceiling (Prop. 8). On our cache panel normalization is at that ceiling, with 226 of 226 predictions at budget four, so perfectly enforced counts would give the same 28.3%. Reviewer MFdY also noted the small image gain of the trimmed-distance score. Section 5.3 now shows that perfect count control could have added at most four certificates there.

Reviewer EYuc found the operation too elementary and asked how widely the setting applies. We agree that the operation is elementary, and we do not claim it. The claims are Theorem 2 and Prop. 8. Theorem 2 is more than a restatement of the substitution, because the same inclusion fails once some sources are protected. Reviewer EYuc has since written that our response addresses part of the concerns, and asked one further question, on content changes that reach preprocessing or training. We have now answered it with a new experiment (Appendix Z of the updated PDF). With one new, correctly labeled image and refitted preprocessing, the nonlinear learner keeps 325 of 400 predictions with normalization and 319 without, against 312 for zero-shot. From two such replacements on, zero-shot is more robust, and the paper says so.

The two weaknesses in your own review were that the theory covered only one cross-class copy with all sources mutable, and that no result existed for multiple replacements. Theorem 2 and the mutable-component result (App. O.2) were added in response to them. If either falls short of what you had in mind, we would like to know. The strengths that your review listed are unchanged. The method, the original experiments and every original number are the same in the revision.

We would be grateful if you could name the specific weaknesses, so that we can address them before the discussion closes.

Reviewer_MFdYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the robustness of few-shot classifiers against support-set poisoning, especially attacks that change class counts by copying support examples. The authors propose fixing the count-dependent coefficients at their initial values to prevent prediction changes caused by class counts. The method is applied to trimmed-distance classifiers and the exponential cache of Tip-Adapter. The paper provides theoretical analysis and robustness certificates under different replacement attacks. Experiments on text and image datasets show improved robustness while preserving the original clean predictions.

Critical strengths
  1. The observation about class-count sensitivity is interesting. The authors identify a specific weakness in support-based classification and propose a simple correction that preserves clean predictions.

  2. The robustness improvement on Tip-Adapter is significant under the single-copy attack setting. The theoretical analysis also provides a useful explanation of this behavior.

Critical weaknesses
  1. The method mainly addresses attacks that change class counts, while many few-shot systems use a fixed number of support samples per class. The paper does not sufficiently justify why this threat cannot be addressed by enforcing fixed class counts.

  2. Although the method improves robustness against single-copy attacks, its certified accuracy under four arbitrary replacements is only 28.3%, compared with 80.4% for zero-shot classification. The improvement on image tasks using trimmed-distance scores is also marginal, raising concerns about the broader practical value of the method.

Questions for authors

See weakness.

2 comments

Thank you for the review. Your first question pointed at something the submission lacked, a comparison against simply enforcing the class counts. The revision adds it as Section 5.6 and Table 3 (changes are in blue), and it also changes how the 28.3% in your second question should be read.

Q1. Why can't this threat be addressed by enforcing fixed class counts?

Enforcing counts is a reasonable policy whenever the system can afford to reject or re-collect data. Normalization does the same job by reweighting. Each class keeps a total weight equal to its reference count, and no record is dropped or refused. We now compare the two directly, and three things come out of the comparison.

First, enforcement cannot buy more robustness than a ceiling that normalization already reaches or nearly reaches. Enforcement stops attacks that change the counts, but it does nothing against replacements that leave every class size unchanged. We prove that any rule which agrees with the original cache when the counts equal the reference counts, whatever it does otherwise (reject, repair or reweight), fails on every count-preserving attack that defeats the original (Prop. 8, App. O.3). We computed this ceiling exactly, with attacks that attain it (Table 3, out of 800 pairs):

budgetcopies: normalized / ceilingarbitrary features: normalized / ceiling
1682 / 683536 / 536
2659 / 661400 / 400
4619 / 623226 / 226

On this panel, normalization is at the ceiling for arbitrary replacements and within four predictions of it for copies.

Second, enforcement has to choose which records to drop, or else refuse to answer. Under every single stored-source copy, normalization keeps 682 predictions correct, order-based fixed-kk repair keeps 677, and rejection with a zero-shot fallback keeps 627 (App. Q.2). The difference between normalization and repair is not statistically significant, so we do not claim that normalization beats repair. It reaches the same level without a drop rule. Rejection has a further problem, because an attacker can force a refusal on any query with one count-changing copy.

Third, normalization needs no trusted snapshot of the counts. In a kk-shot task the reference is kk, or the received total divided by the number of classes (App. Q.1).

Q2. 28.3% certified under four arbitrary replacements, against 80.4% for zero-shot

The 226/800 (28.3%) equals the ceiling in the table above. Perfectly enforced counts would leave the same 226 predictions, so better count handling cannot close this gap.

The gap comes from admitting four adversarial features into classes of ten records. Zero-shot never reads the support set and cannot be poisoned through it, and any adapter that does read the supports pays this cost. Under the copy threat that motivates the paper the picture is different. Normalized Tip-Adapter stays above zero-shot up to three copies (682, 659 and 645 against 643) and falls below it at four (619), and it keeps the clean gain of the cache (700 against 643 correct, +13.8 points on EuroSAT).

Beating zero-shot under arbitrary replacements takes more supports. A median-affinity rule with seventeen supports per class, which we fixed before a source-disjoint confirmation on 6,400 new images, keeps 4857 of 6000 pairs correct under four arbitrary replacements, against 4713 for zero-shot (+2.4 points, 95% CI [1.27, 3.52]). It still trails zero-shot on CIFAR-FS by 26 of 3000 (App. R). Separately, when one poisoned cache serves all queries of an episode, normalization keeps at least 416 of 800 correct, and the original at most 224 (App. X).

Q3. The image gain of the trimmed-distance score is marginal (122 → 124)

We agree that it is small, and the revision now explains where the small number comes from (new paragraph in Section 5.3). For this score there was little to gain from fixing counts in the first place. Even perfectly enforced counts would add only four image certificates (H4=126−122H_4=126-122). Normalization recovers two of them, and on text it recovers all 36. A tighter verifier would not raise 124 either, because each of the other 79 clean-correct image predictions has an explicit four-replacement attack (App. O.4). 124 is the exact robust count for this score.

Where counts carry more weight, the gain is large. With an additive anchor-plus-support distance, which is a different score from the submitted one, normalization raises image certificates from 24 to 145 of 300 (ceiling 147), and from 27 to 178 of 400 on source-disjoint images (App. O.4–O.6, O.9). For the submitted score, image copy harm still falls by 42% (31 → 18), and it falls for every encoder (Table 1).

We hope this clarifies both the comparison with count enforcement and the gap to zero-shot. We are happy to answer follow-up questions.

Dear Reviewer MFdY,

The discussion period closes on November 18 (AoE), so we wanted to ask whether our response answers your concerns. The short version is this. Enforcing fixed class counts cannot give more robustness than a ceiling that we prove in the revision (Prop. 8). On our cache panel, normalization is already at that ceiling under arbitrary replacements, with 226 of 226 predictions at budget four, which is the 28.3% you asked about. Under copies it is within four predictions of the ceiling. The comparison is in the new Section 5.6 and Table 3. The gap to zero-shot under arbitrary replacements is real and the paper states it, but perfectly enforced counts would leave the same gap.

If any of this is unconvincing, or if another concern remains, we would be glad to respond in the time that is left. Thank you again for your review.

Comments on the paper

2 comments

We thank all three reviewers for their time and their detailed comments. We were glad that the reviewers found the count-sensitivity observation interesting (MFdY, W8Fg), the correction simple and exactly clean-preserving (all three), and the evaluation careful and honestly reported (W8Fg, EYuc).

A revised PDF is uploaded. Everything that changed relative to the submission is printed in blue, with a margin note naming the review comment it answers, and a one-page guide before Appendix A lists where each comment is addressed. The main text uses the tenth page allowed during the discussion period. Detailed replies are posted under each review. Below we summarize the four concerns that shaped the revision.

1. Why not simply enforce fixed class counts? (MFdY; new Section 5.6, Table 3)

The submission did not answer this question, and the revision now does. A count fix can only achieve so much. Any rule that agrees with the original predictor whenever the class counts equal the reference counts inherits the original's failures under attacks that leave the counts unchanged (Prop. 8). This holds for enforcing counts, for repairing them and for rejecting the input. On our cache panel, normalization already reaches this ceiling (out of 800 query pairs):

budgetcopies: normalized / ceilingarbitrary features: normalized / ceilingzero-shot
1682 / 683536 / 536643
4619 / 623226 / 226643

The 28.3% certificate that MFdY asks about (226/800) is therefore what perfectly enforced counts would also give. It remains well below zero-shot, which never reads the supports. Closing that gap takes more support information: with seventeen supports per class and a median rule fixed before a source-disjoint confirmation, 4857 of 6000 pairs stay correct under four arbitrary replacements, against 4713 for zero-shot (App. R). Normalization also keeps predicting without discarding or refusing data. Under every single copy it keeps 682 predictions correct, compared with 677 for fixed-kk repair and 627 for rejection with a zero-shot fallback.

2. Theory beyond a single copy (W8Fg, EYuc; new Theorem 2)

The submission proved robust-set inclusion for one cross-class copy. Theorem 2 extends it to every budget below the initial class sizes when all sources are mutable. Every prediction that the original cache keeps under multiple same- or cross-class copies, or under arbitrary bounded-feature replacements, is also kept after normalization, and the theorem gives the original's exact worst case. With protected sources, the counterexample in the paper still applies to full-class normalization. Normalizing only the mutable part of each class restores inclusion there (App. O.2), and App. W gives conditions for other count coefficients.

3. The contribution (EYuc; new "Contributions" paragraph in Section 1)

We agree with EYuc that the operation is elementary, and we do not claim the operation as the contribution. We claim four results about it. With all sources mutable, robustness never decreases. This does not follow from clean preservation, since the same operation loses robustness once some sources are protected. No clean-preserving count correction can exceed the ceiling above, which normalization reaches on our cache panel. The certificate covers the complete modified scores. Finally, changing class counts is a poisoning route that count-preserving few-shot certificates do not model, and normalization closes it.

4. Deployment, the count snapshot, and contamination outside the cache (W8Fg, EYuc; Section 3.1 and Section 6)

No trusted snapshot is needed. The reference count is part of the task design. It is the shot count kk or, if kk was not stored, the received total divided by the number of classes (App. Q.1). Records are encoded independently, so contamination at collection time is the same replacement as contamination of the cache (App. N.5). When the supports also feed preprocessing or training, applying the reference weights before any support-dependent step keeps those computations unchanged under count changes. We verified this with refitted preprocessing (App. P.4) and with full visual-encoder training (App. U).

Limitations. Three limits remain, and the paper states each of them. Normalization removes the count effect only, so poisoning that changes the content of a class is outside its guarantee. With ten supports per class and arbitrary feature replacement, zero-shot remains more robust than the normalized cache (point 1). The image gain of the trimmed-distance score is small, and Section 5.3 now shows that perfect count control could have added at most four certificates there.

We thank the reviewers again and are happy to answer further questions during the discussion.

We uploaded a second revision of the PDF. It differs from the revision of November 13 in two places, both in blue.

  1. A new Appendix Z answers Reviewer EYuc's follow-up question on poisoning that changes the content of the support set and reaches preprocessing or training. It contains a bound on how far the weighted support distribution can move (Prop. 15) and a new experiment in which stored records are replaced by new images and both the shared preprocessing and the adapter are refitted after every replacement (Table 43). Normalization still helps there and never reduced robustness, but its gain is limited by content, and from two replacements on zero-shot is more robust. Section 6 now states this result in one sentence.

  2. To keep the main text within ten pages, we shortened three sentences in Section 5.6. No number or claim changed.

Nothing else changed. All numbers, theorem numbers and table numbers cited in our earlier responses are the same in the new PDF. The supplementary material contains the code and the per-support records of the new experiment.

We have replied to Reviewer EYuc's follow-up question and to Reviewer W8Fg's comment in their threads. Reviewer MFdY, if any point in our response remains open, we would be glad to answer it before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.