- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies support-set poisoning in frozen-feature few-shot classifiers when source replacements change class counts. It separates count-dependent coefficients from support statistics and fixes those coefficients at their pre-attack values. The correction is applied to trimmed-distance scores and Tip-Adapter caches. The authors establish clean-prediction preservation, derive certificates for bounded-feature replacements and exact margins for single-copy attacks, and evaluate the modified classifiers on text and image classification tasks.
Critical strengths
The correction is inexpensive and preserves the original clean predictions exactly. The paper also evaluates its consequences carefully through certificates, exhaustive single-copy enumeration, and shared-cache attacks with held-out queries. These results substantiate that the modification can reduce the measured vulnerability.
Critical weaknesses
-
The central methodological contribution is too elementary to provide sufficient technical novelty. The essential operation is to express a class sum as the current class count multiplied by a class mean, then replace the current count with its pre-attack value. In Equation (8), this amounts to applying the classwise factor . The trimmed-distance specialization similarly freezes the initial retained-count coefficient. Clean-score preservation follows directly because the initial and current counts coincide before an attack. In my view, this coefficient substitution does not constitute a sufficiently substantial new adaptation or robustness method. The general coefficient-freezing formulation largely restates the same operation.
-
The assumed attack setting has limited general applicability. The strongest empirical results concern copying already stored, correctly labeled support records within a frozen cache. Although the paper also provides certificates for broader bounded-feature replacements, those guarantees remain conditional on fixed feature computation and a trustworthy initial count snapshot. The practical relevance of this interface is insufficiently established, particularly for pipelines in which contamination occurs during support-set construction or influences preprocessing or model adaptation. The experiments therefore demonstrate robustness within a specific interface without establishing how broadly the improvement transfers to realistic poisoning scenarios.
Questions for authors
-
What is the principal methodological or theoretical contribution beyond freezing the pre-attack count-dependent coefficients?
-
What concrete deployment scenario motivates the assumed attacker access and trustworthy initial count snapshot? How would the proposed correction apply when contamination occurs before that snapshot is established?
4 comments
Dear Reviewer EYuc,
The discussion period closes on November 18 (AoE), and we would value your view on whether our response addresses your two questions. On the contribution, we agree that the operation is elementary. What the revision puts forward are results about it. Theorem 2 shows that normalization never loses robustness at general budgets when all sources are mutable. The same statement fails once some sources are protected, so it does not follow from the substitution itself. Prop. 8 (Section 5.6) bounds what any clean-preserving count correction can achieve. On deployment, no trusted count snapshot is needed, because the reference count is the shot count fixed by the task design (Section 3.1, App. Q.1).
If the soundness score reflects a specific proof or experiment that you consider incorrect, please point us to it and we will respond right away. Thank you again for your time.
We thank the authors for their response and the additional theoretical results, which address part of concerns. However, it remains unclear how far the robustness results extend to settings where changes to support-set content affect preprocessing or training.
Thank you for reading our response and for the follow-up. The remaining question is a fair one, and we ran a new experiment to answer it. The guarantees extend exactly as far as class counts do. Past that point they do not hold, and no correction based on counts can make them hold. We measured what happens there. The updated PDF reports it in a new Appendix Z, and Section 6 states the outcome.
The guarantee covers every change that alters the class counts but leaves the records of each class as they were, for example when a whole class is repeated. For such changes, a pipeline that applies the reference weights before every support-dependent step has the same deterministic preprocessing and the same fixed-step training trajectory as the clean run (Prop. 7, App. O.1).
It stops when a replacement brings new content into the support set, because the training data itself has then changed. Replace one record by a different record of the same class. Every count stays the same, so a correction that looks at counts has nothing to act on, yet the shared statistics move. We now bound this movement. In our five-way ten-shot design, the weighted support distribution moves by at most 0.038 in total variation after one replacement and 0.153 after four (new Prop. 15). This limits how far a shared mean can move. It is too coarse to certify the predictions of a refitted learner, and we do not claim such a certificate.
We therefore ran the experiment that your question describes. Stored records were replaced by images that are not in the support set, and after every replacement the shared preprocessing was recomputed and the adapter refitted. The two learners are those of App. O.5: ridge after centering, and ridge after centering and unit normalization, which is a nonlinear shared step. For one replacement the enumeration is complete, over every stored position and every candidate image. For two to four replacements we report attacks found by a beam search, so those numbers are upper bounds on robustness. The table gives robust-correct predictions out of 400 source-disjoint queries when the inserted image carries its correct label. Zero-shot keeps 312.
| learner | clean | 1 replacement: uncorrected / normalized / ceiling | 2: uncorrected / normalized | 4: uncorrected / normalized |
|---|---|---|---|---|
| ridge | 355 | 302 / 328 / 332 | 209 / 290 | 80 / 198 |
| nonlinear preprocessing | 372 | 319 / 325 / 335 | 262 / 282 | 145 / 196 |
The ceiling is the most that any count correction can reach (Prop. 8). We read three things from the table.
First, normalization still helps when preprocessing and the adapter are refitted on new content, and it never hurt. With one replacement, where the enumeration is complete, no prediction that survived without it was lost with it. The search found no such case at larger budgets either.
Second, what remains is mostly content. For the nonlinear learner, replacements that change no count already take the 372 clean-correct predictions down to 335. Of the 16 predictions that a count correction could still recover, normalization recovers 6. For ridge it recovers 26 of 30.
Third, the range is short. With one new image the refitted learners keep 325 and 328 predictions against 312 for zero-shot. This difference is not statistically resolved, and it comes from EuroSAT alone. From two replacements on, both learners fall below zero-shot. If the attacker may also choose the label, the nonlinear learner is below zero-shot already at one replacement (263). The 800-pair cache panel with two encoders shows the same pattern (Table 43).
So the direct answer is this. When several records with new content can reach preprocessing or training, we have no guarantee beyond removing the count effect. Normalization still improves on the uncorrected learner there, but zero-shot or a larger support set is the safer choice. For full visual-encoder training we have only the trajectory-level evidence of App. P.5, where a content-changing replacement leaves a nonzero parameter difference under every correction. The paper's claims concern the count effect, and the revision now states this boundary with numbers.
We hope this settles the remaining point. If it does, we would be grateful if you would consider updating your assessment. We are glad to answer anything further before the discussion closes.
Thank you for the detailed review, and for noting that the correction is inexpensive, preserves clean predictions exactly, and is evaluated carefully. Changes in the revised PDF are in blue.
Q1. What is the contribution beyond freezing the count-dependent coefficients?
We agree that the operation is elementary and that clean preservation follows at once. That is intended, since such a correction can be added to an existing predictor at no cost. We also do not claim the operation as new. It has the form of prior-ratio weights (Lipton et al., 2018), which the revision now cites. What we claim are three results, none of which follows from clean preservation.
(1) Normalization never loses robustness at general budgets (new Theorem 2). Take an all-mutable cache and any budget m below the initial class sizes. Every prediction that the original keeps under all m copies, or under all m arbitrary bounded-feature replacements, is also kept after normalization, and the theorem gives the original's exact worst case. This is not automatic. The same operation loses robustness once some sources are protected (exact counterexample, minimum gap 1/70 → −9/770), and it fails for some other count coefficients (App. W). A variant that normalizes only the mutable part restores inclusion with protected sources (App. O.2), and App. W gives the conditions for general coefficients.
(2) A ceiling for every clean-preserving count correction (new Section 5.6, Table 3). No rule that keeps the original predictions whenever the counts equal the reference counts can beat the original's robustness under count-preserving attacks, and this includes enforcing fixed counts (Prop. 8). On our cache panel, normalization meets this ceiling under four arbitrary replacements (226 of 226, out of 800 pairs) and comes within four predictions of it under four copies (619 vs. 623). Reaching the ceiling is an empirical finding on this panel, not a theorem. It tells us that the remaining failures come from content, and fixing those needs a different predictor.
(3) A poisoning route outside existing few-shot certificates. FCert and LeFCert certify threats that keep the class counts fixed (L100–103). Correct-label copies act through the counts alone (Fig. 1). In native Tip-Adapter, single copies flip 325 of 700 correct predictions, and normalization reduces this to 17.
Q2. Deployment scenario, the count snapshot, and contamination before the snapshot
The scenario is now described in a new paragraph of Section 3.1. A task owner fixes the class vocabulary, the encoder and the k-shot design, and a separate collector or a writable record store supplies the records. SuS-X, for example, builds labeled caches from external URLs whose content can change after indexing (Carlini et al., 2024). The attacker controls some records but not the configuration. We do not claim an attack on a deployed system.
No snapshot is needed. The reference count is part of the design, n0,c=k in a k-shot task. If k is not stored, the reference is the received total divided by the number of classes, which replacement leaves unchanged (App. Q.1). On 5,104 received inputs the scores match those obtained with known k. Unknown class proportions cannot be identified from counts alone, and we state this as a limit.
Contamination before any cache exists is the same replacement, because records are encoded independently. Poisoned raw manifests encoded from scratch give identical features, scores and predictions (App. N.5).
The harder case is contamination that reaches preprocessing or adaptation (Section 6, "Beyond frozen features"). If records are weighted by n0,c/nc before any support-dependent computation, the class-weighted empirical measure does not change when counts change, so deterministic shared preprocessing and fixed-step training are preserved (App. O.1). We tested this with actual refitting and training:
Poisoning that changes content remains outside these guarantees, as Section 6 states.
Presentation. We added a contributions paragraph, moved the key results into the main text, and added a reviewer guide before Appendix A.
We hope this answers your concerns and that you will reconsider your assessment. If there is a specific correctness problem behind the soundness score, please tell us and we will address it.