- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces an attack, KEEP, that recovers an image classifier’s domain and class names using only its top-1 label and confidence score. KEEP first probes the model with about one million captioned LAION and WIT images, then uses high-confidence captions to infer the domain. It builds a domain-specific probe set and names each class from its 20 highest-confidence captions.
On ImageNet-1K, KEEP reaches 0.49 exact-label accuracy and about 0.9 lexname accuracy. Across 14 fine-tuned classifiers, it outperforms CLIP-Dissect on 10, despite CLIP-Dissect receiving the true vocabulary. Hiding confidence scores largely prevents domain inference, while rounding or adding noise does not.
Critical strengths
- It's novel to treat a classifier's label space as the leakage target.
- The proposed method requires no shadow models, gradients, or optimization.
- The evaluation is broad. The fourteen victims cover four architectures and diverse domains. The ablations over pool size, number of retained probes, probe selection, and namer size show that confidence-based selection drives recovery .
- The finding that rejecting low-confidence inputs strengthens the attack rather than weakening it is counterintuitive, but valuable.
Critical weaknesses
- All victims are fine-tuned on public benchmarks, and GPT-5.6 explicitly identifies "Food-101-style," "Places365," and "NWPU-RESISC45" as domains. So it is possible that the lexical roster benefits from the LLM's memory of these taxonomies. And LAION and WIT may contain the benchmark images used to train the victims, which would increase confidence on memorized images.
- The celebrity victim recognizes famous people whose names appear the most in web captions, while a face recognition model covering private individuals offer no captioned evidence. In the intellectual-property framing, KEEP recovers which concepts a model recognizes, not which data it was trained on, and no training samples are identified or reconstructed.
- In the lexical setting, KEEP first asks an LLM for possible class names and then searches for captions containing those names. The paper does not compare against simple alternatives, such as selecting frequent phrases from the top captions or using a vision-language model to describe the top images. The comparison with CLIP-Dissect is also unclear because KEEP uses a targeted image pool while CLIP-Dissect uses the original pool.
Questions for authors
- How do you rule out benchmark memorization by the LLM or overlap between the victim’s training data and the LAION/WIT probe pools? Could you test KEEP on a private taxonomy and a deduplicated probe set?
- Does KEEP reveal training-data information or only the concepts a model recognizes? How would it perform on private identities or classes with little captioned web data?
- Can you compare KEEP with simple caption-frequency and VLM-captioning baselines under the same probing pool?
4 comments
W2 / Q2 (a). KEEP reveals that a class was part of the training data, and not individual training samples.
We agree that KEEP recovers the concepts a model was trained to recognize, and that it does not identify or reconstruct training samples. Sample-level attacks such as membership inference and model inversion address that question, and they rely on the domain or class knowledge that KEEP recovers (Section 2). A classifier has an output class only because it was trained on examples of it, so the recovered vocabulary is information about the training data at the level of the class. As argued in Section 1, this is the question at stake for a rights holder who can only query the deployed model.
W2 / Q2 (b). Recovery decreases with the amount of captioned web data, and a pool collected by the attacker restores it.
We agree that KEEP depends on the availability of captioned evidence, and we measured how recovery varies with it. For each of the 3,415 classes of the fourteen victims, we counted the captions that name it among the 106.7M captions of our retrieval corpus. Recovery is 40% for the classes named in fewer than ten captions, 75% for those named in ten to 99, and 70% above. For the celebrity victim, it is 7% for the identities named in fewer than 100 captions, 53% between 100 and 999, and 80% above 10,000. We therefore acknowledge that identities with little web presence, and private individuals in particular, are not recovered from web pools, in which case KEEP abstains (Section 3).
However, once the domain is inferred, the pool does not need to come from the web (Table 3). To test this for identities, we replaced the web pool with a gallery of named face photographs (VGGFace2, 20 images for each of its 9,131 identities), in which the only text attached to an image is a name. The gallery is not built from the victim's classes, and contains 223 of its 997 identities. From this gallery, KEEP names 214 of the 223 (96%), against 134 (60%) from the lexical web pool, including 85 of the 89 that the web pool misses. The identities absent from the gallery receive a wrong name, but the two cases are separable from the evidence itself: retaining only the classes for which one name covers at least half of the selected probes gives 212 predictions with a precision of 90%. These identities are public figures and serve as a stand-in; we did not collect data of private individuals. The experiment nevertheless suggests that, for a face-recognition model, the limiting factor is whether the attacker can obtain named photographs of the candidates (e.g., from professional or social-network profiles), rather than the captions available on the web.
W3 / Q3. KEEP outperforms a caption-frequency rule and a vision-language model given the same probes.
We thank the reviewer for suggesting these baselines. In both, every method names a class from exactly the same evidence, namely its k = 20 most confident probes, on the unfiltered pool and on the lexical pool. The caption-frequency baseline returns a phrase from the captions of these probes without a language model; we report, for each victim, the best of three rules (the most frequent phrase, the most distinctive phrase under class-based TF-IDF, and its longest variant). The vision-language baseline (Qwen2.5-VL-7B, of the same family as our namer) is shown the 20 images with the prompt of KEEP and never sees their captions. All rows are scored as in Figure 6.
| Recovery (%) | Food | Dogs | Flowers | Pets | Sneakers | Aircraft | Birds | Places | Logos | Cars | Pokémon | Celeb. | Aerial | Skin |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Unfiltered pool: KEEP | 71 | 68 | 47 | 78 | 54 | 35 | 15 | 43 | 41 | 41 | 7 | 12 | 27 | 3 |
| Caption frequency | 36 | 18 | 24 | 27 | 16 | 13 | 2 | 12 | 9 | 13 | 0 | 1 | 11 | 0 |
| Vision-language model | 64 | 61 | 55 | 73 | 46 | 23 | 10 | 48 | 54 | 38 | 2 | 15 | 38 | 6 |
| Lexical pool: KEEP | 98 | 93 | 85 | 86 | 82 | 79 | 78 | 69 | 63 | 68 | 69 | 55 | 36 | 29 |
| Caption frequency | 82 | 78 | 75 | 70 | 68 | 47 | 46 | 51 | 41 | 51 | 51 | 42 | 38 | 10 |
| Vision-language model | 91 | 76 | 75 | 84 | 68 | 34 | 43 | 48 | 50 | 57 | 43 | 43 | 31 | 16 |
Caption frequency. KEEP outperforms the frequency rule on all fourteen victims on the unfiltered pool, where the rule recovers at most 36% of the classes, and on thirteen on the lexical pool (aerial scenes being the exception, 38% against 36%). The difference is smaller on the lexical pool because every caption there was retrieved by a roster name, which is consistent with our response to Reviewer LNYR: a large part of the recovery comes from the targeted pool, which is itself a product of the domain inferred by KEEP.
Vision-language model. On the lexical pool KEEP outperforms the vision-language model on all fourteen victims, with the largest differences where the class is a specific named entity (79% against 34% on aircraft, 78% against 43% on birds, and 69% against 43% on Pokémon). On the unfiltered pool KEEP is ahead on eight victims and the vision-language model on six (flowers, places, logos, celebrities, aerial scenes, and skin lesions), i.e., where the web captions of the probes do not name the class but the images depict it. We see this as complementary to KEEP, in line with the extension discussed in Section 5, and note that on every victim except aerial scenes the vision-language model on the unfiltered pool remains below KEEP on the lexical pool. As a further control, we gave our namer the captions written by the vision-language model for the five most confident probes of each class instead of their web captions: on the twelve victims we could run, the web captions give higher recovery on ten and the same on two.
CLIP-Dissect. We agree that Figure 6 does not separate the pool from the naming procedure. The comparison on the same lexical pool is reported in our response to Reviewer LNYR (Q3 (c)): with the same evidence, KEEP outperforms CLIP-Dissect with the ground-truth vocabulary on seven of the fourteen victims and matches it on one, and outperforms it on eleven when CLIP-Dissect is given the roster R, the only vocabulary available to the attacker.
Thanks for your response, which has addressed my concerns. However, considering the overall novelty and quality of the manuscript, I maintain my current score.
We thank the reviewer for the constructive assessment and for the concrete experiments suggested, all of which we have run. We address each point below. All new experiments use the pipeline of the paper, with the same prompts and settings.
W1 / Q1 (a). The language model recalls the class list of two victims only, and KEEP recovers class lists that no model could have memorized.
We agree that GPT-5.6 recognizes the benchmark behind three victims (Table 5), and we measured how much this contributes. If the roster R were recalled from memory, its entries would be the victim's classes. This is the case for aerial scenes, where all 45 entries are class names, and largely for places (61%). In contrast, only 7% of the 1,461 entries generated for the food victim are Food-101 classes, despite the domain being described as "Food-101-style", and the share is at most 18% for nine of the remaining victims (37% for celebrities and 62% for Pokémon, whose classes are most of the existing species). The roster is therefore an enumeration of the inferred domain, from which the victim's confidence selects the relevant entries. We also repeated the lexical attack on the three victims with the domain and roster produced by a smaller language model (GPT-5.4-mini) that does not recognize any benchmark: recovery is 80% for food, 57% for places, and 33% for aerial scenes, against 98%, 69%, and 36% in the paper. We note that this run also used a smaller retrieval corpus, so it is a lower bound. Finally, the semantic pool does not use the roster at all, and still recovers 89% of the food classes, 90% of the dog breeds, 70% of the logos, and 62% of the Pokémon (Table 6).
Private taxonomy. Training a classifier on a private taxonomy was not feasible within the discussion period, so we constructed victims whose label set is private. We take a public ImageNet-21K classifier (ViT-B/16) and discard all outputs except a random subset of the ImageNet-21K classes of one domain that are not ImageNet-1K classes. To the attacker this is a classifier with opaque identifiers, and its class list is a random draw that appears in no benchmark. We ran the full attack on three such victims, namely the three domains with the densest caption evidence in a pilot over ten candidate domains.
The domain is inferred correctly for the three victims, and KEEP recovers between a third and a half of class lists that the language model cannot have memorized. The remaining classes are mostly rare WordNet names (e.g., "cathedra", "box coat"), for which the predicted name is a broader category such as "chair" or "jacket". We acknowledge that only the label set of these victims is private, as the underlying model was trained on public data.