- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces SimPrint, a method to fingerprint language models using multiple questions and answers. The method first fixes a code-word derived from an error correcting code. Then it maps each bit of the code-word to binary questions, whose answers are determined (mostly) by the code-word bit. The question-answers are then injected into the LLM using a knowledge editing method inspired by AlphaEdit. The paper demonstrates the effectiveness of the scheme on three models, as well as the robustness to fine-tuning, merging, pruning and quantization. The method also leads to negligible drop in the utility of the model.
Critical strengths
- The method proposed is very nice. I like the idea of using coding theory to come up with code words that can survive some amount of degradation or corruption induced by model modifications. This seems like a very natural idea.
- The paper is quite well-written and the method is easy to follow.
- I liked the False positive rate analysis, though I would have liked to see some empirical evidence as well.
Critical weaknesses
My main issue with the paper is the empirical evaluation, especially wrt prior work
- While I like the idea of using multiple queries for establishing ownership, Semantic Watermarking (Gloaguen et al 2026) has similar flavour as well. Indeed, from Table 1, it seems that RF has very similar performance to the proposed method except under model merging. However it is unclear how many queries and what p-value threshold was used for this method --- ideally one needs to match either number of queries (i.e. 192), false positive rate or utility degradation among different methods. Similarly for Scalable Fingerprinting, the clean fingerprinting rate itself is very low (10% on Qwen3). This indicates that the hyper-parameters were not tuned sufficiently for this method, since the clean rate should be 100% for a well-tuned baseline. It is also unclear to me how the FPR for SF. IF or CTCC are computed --- are the fingerprints treated independently, or is a model said to be fingerprinted if greater than k% of fingerprints fire?
1.1 Scalable fingerprinting (Nasery et al 2025) provides an approach to answer the question from lines 63-65 of ownership from partially recoverable model behaviour. Even chain-and-hash (Russonivich&Salem 2025) achieves a similar goal successfully. Hence, I believe that Contribution 1 overclaims being the first work to address this question.
1.2 It is unclear if the experiments only involve base models or instruct models as well.
- For results under attack, particularly serving time perturbations, output perturbations (Nasery et al 2026) do not seem to be considered. I belieive that the attacks (SuppressTopK, SuppressNeighbours) would achieve high ASR against the present scheme. In fact there are no adaptive attacks on the system at all. While this is not crucial, it would strengthen the paper to know what the practical bottlenecks of the scheme are.
Refs LLM Fingerprinting via Semantically Conditioned Watermarks, Gloaguen et al, ICLR 2026 Scalable Fingerprinting of LLMs, Nasery et al, NeurIPS 2025 Hey, that's my model! Introducing Chain-and-Hash, Salem & Russonivich, ICLR 2026 Are robust llm fingerprints adversarially robust?, Nasery et al, SaTML 2026
Questions for authors
- Does the method survive output pertubations (e.g. Suppress-top-k from Nasery et al 2026)?
- Can the experimental details for the baselines (e.g. learning rates, epochs for backdoor based methods, number of queries/p-value threshold for RF), as well as the other evaluation settings be reported?
4 comments
Response to W2 and Q1-2
W2 & Q1. Output Perturbations and Adaptive Attacks
We thank the reviewer for highlighting output perturbations during model serving. We evaluated 16 fixed and 12 confidence-selective attack configurations on two fingerprinted models and explored verification improvements without retraining or additional model queries.
(1) Attack Settings.
We evaluated two attacks on Qwen3.5-9B and Llama-3-8B-Instruct:
- SuppressTopK: Sets the logits of the top- highest-probability tokens to negative infinity at the first candidate continuation positions.
- SuppressNeighbor: Additionally suppresses morphological-prefix neighbors of high-probability tokens using a fixed suppression set at the first positions.
With and , these yield 16 fixed configurations.
For confidence-selective attacks, candidate confidence is computed from the original mean negative log-likelihood (NLL):
Perturbation () is applied only when , with , yielding 12 additional configurations.
(2) Suppression-Aware Verification and List Decoding.
The original SimPrint verifier failed under all 16 fixed configurations. However, hard suppression can disrupt semantic decisions without necessarily destroying the underlying identity information. We therefore explored three strategies requiring no retraining or additional queries:
- Erasure Verification: Mark a bit as unknown if either candidate is suppressed, then apply RS erasure decoding.
- Suppression-Aware Verification: Retain the original decision when both scores are finite; infer the bit from suppression of the more probable candidate when only one is suppressed; otherwise mark it as unknown.
- Suppression-Aware + List Decoding: Enumerate unknown-bit assignments and apply RS(24,8) decoding. Match against registered identities only if all successful decoding outcomes yield a unique identity.
Enumeration is limited to 12 unknown bits (4,096 local decodings); larger cases are rejected. Neither enumeration nor filtering uses the expected identity, and the search never terminates early upon finding a registered identity.
(3) Identity Recovery Results.
The table reports exact identity recovery successes.
| Attack | Original | Erasure | Suppression-Aware | Suppression-Aware + List |
|---|---|---|---|---|
| SuppressTopK | 0/8 | 2/8 | 8/8 | 8/8 |
| SuppressNeighbor | 0/8 | 0/8 | 2/8 | 8/8 |
| Confidence-Selective | 8/12 | 12/12 | 12/12 | 12/12 |
| Clean | 2/2 | 2/2 | 2/2 | 2/2 |
Suppression-aware verification recovered all identities under SuppressTopK and confidence-selective attacks; adding list decoding recovered the six remaining SuppressNeighbor failures.
These Neighbor cases contained 1–11 unknown bits, requiring at most 2,048 enumerations, with exactly one distinct identity candidate per exhaustive search. Additionally, no false acceptance was observed in 13 historical clean records from unsigned models (0/13).
These results show that verification failure under output suppression does not necessarily imply identity loss. Suppression-aware extraction and bounded list decoding recovered all identities under the tested attacks without additional model queries. We will clarify the attack interface, decoding rules, and experimental limitations in the revision.
Q2. Baseline Hyperparameters and Reproducibility Details
We summarize baseline configurations and verification protocols, following the original papers or official implementations.
| Method | Training Configuration | Verification Protocol |
|---|---|---|
| IF | IF-Adapter: LR , 20 epochs, adapter dim 16, effective batch 48; IF-SFT: LR , 3 epochs, effective batch 64 | 10 trigger queries, greedy decoding |
| SF | LR , 30 epochs, batch 8, 1,024 training fingerprints | 128 trigger queries |
| CTCC | LoRA, LR , 12 epochs, rank 8, alpha 16, effective batch 16 | 95 trigger queries; temperature 0.95, top-p 0.7, max tokens 512 |
| RF | Adafactor, LR , 2,500 steps, batch 64, 250 warmup steps, cosine schedule | 1,000 French Alpaca queries; Z-test ; 5 independent decisions |
| iSeal | LR , 15 epochs, batch 2, 10 training samples | 10 plaintext query samples |
We will provide complete checkpoint names, training configurations, query budgets, aggregation rules, and statistical definitions in the revision to support reproducibility.
I thank the authors for their detailed response. There are a few comments I would like to make ---
RF query budget, model utility
Thanks for the clarification on verification for RF. I also like the idea of iso-utility comparison (with the caveat being how one defines utility, since some methods and settings would be better at certain datasets). However, from the table in the rebuttal, it seems that RF still dominates SimPrint in the utility-fingerprinting trade-off, with model merging being the only robustness gain that SimPrint offers.
SF Clean FSR. SF's 10% Clean FSR on Qwen3-8B
Note that using the same hyper-parameters from the SF paper with a different model (afaict the original SF work did not evaluate on Qwen3 at all) is not a fair comparison. Ideally one needs to tune the hyper-parameters for each method on each model considered.
FSR and aggregation
Sorry, the FPR in my original review was a typo, I did mean to ask about fingerprint success rate. It is still unclear to me how the aggregation works for other baselines, for example for IF is something verified only if all 10 fingerprint responses are correct? The table in the second response does not seem very clear to me.
Adaptive attacks
Thanks for this additional experiment. It appears that SimPrint can be easily bypassed by output perturbations. I am unsure of the mitigations described in the rebuttal, could the authors write them out more clearly? For example, what does "Retain the original decision when both scores are finite; infer the bit from suppression of the more probable candidate when only one is suppressed;" mean? Does one require access to logits for this? The new verification protocols appear to be an extension of the original method and probably require closer scrutiny, hence I would like it if the authors described them as clearly as possible. I also do not understand what is meant by "verification failure under output suppression does not necessarily imply identity loss."
Overall I think my original concerns with the paper still remain and hence I would keep my rating.
We thank Reviewer kYZ6 for the follow-up. We re-verified our logs and verification code, and respond below.
1. RF Query Budget, Utility, and Trade-offs Model merging is where SimPrint's advantage is most pronounced, but it is neither the only robustness gain nor the only practical advantage over RF:
-
Beyond Merging: On Llama-3-8B fine-tuned on Alpaca (Table 1), SimPrint recovers the identity (100%), whereas RF fails all five tests (0%; ). On Qwen3-8B (Table 1), RF also drops to 70% / 0% under Random Pruning (0.1 / 0.2) and 80% / 60% under Taylor Pruning (0.1 / 0.2), while SimPrint keeps 100%.
-
Overall Trade-off: Trade-offs should jointly consider transformations, query budgets, and identity recovery. For a pp average utility difference ( vs. pp), SimPrint uses fewer queries per test (192 probes vs. 1,000 queries), achieves exact 64-bit identity recovery, and maintains 100% robustness where RF degrades.
2. SF's Clean FSR on Qwen3-8B While tuning SF for Qwen3-8B could raise its Clean score, the observed robustness gap does not stem solely from Qwen3-8B:
-
On Llama-3-8B and Llama-3.2-3B, SF reaches 88.2% and 88.6% Clean matching, yet incurs large utility drops ( and pp) and still collapses to 0%–4.1% after TIES and DARE+TIES merging (Tables 1 & 8).
-
Thus, merging severely disrupts SF even when initial implantation is strong. We will clarify that 10.45% on Qwen3-8B is the default-setting result rather than SF's tuned upper bound.
3. FSR Definitions and Aggregation For IF, no: it does not require all 10 responses to be correct. Table 1 reports the triggered fraction across the 10 queries (e.g., 8/10 on Llama-3-8B after Alpaca is 80%), without converting partial scores into binary model-level pass/fail decisions.
| Method | Metric in Table 1 | Unit |
|---|---|---|
| IF | Fraction of 10 queries containing target string | Per-query retention |
| SF | Fraction of 128 queries exactly matching registered tokens | Per-query retention |
| CTCC | Fraction of 95 queries containing target marker | Per-query retention |
| RF | Fraction of 5 tests (1,000 queries/test) passing | Model-level detection |
| iSeal | Fraction of 10 queries achieving | Per-query retention |
| SimPrint | Exact match () of 64-bit identity after RS decoding | Complete identity recovery |
Thus, IF, SF, and CTCC retain partial per-query credit, whereas SimPrint is evaluated under a stricter criterion requiring complete, exact identity recovery. We will clarify this in Section 4.1.
4. Output Perturbations and Extended Verification
(1) Logit access: Yes. This defense extension requires logits/candidate likelihoods to detect whether preset answers are hard-suppressed. It applies to interfaces exposing scores (e.g., logprobs), not strictly text-only APIs. We will state this explicitly.
(2) How the new rules work: Hard suppression sets top-token probabilities to zero, giving any candidate containing them zero probability. When suppression occurs:
| Post-Attack Suppression | Bit Reading Rule |
|---|---|
| Only one candidate suppressed | Select the bit of the suppressed candidate. |
| Both candidates suppressed | Mark bit as unknown (?). |
- Why Rule 1 works: Because the attack only suppresses high-probability tokens, a single suppressed candidate must have been the higher-probability choice before perturbation (no pre-attack ranking needed). This rule alone (
Suppression-Aware) recovers 8/8 identities underSuppressTopKand 12/12 underConfidence-Selective. - Bounded Enumeration (
+ List Decoding): For remaining unknown bits ( allowed, i.e., local decodings; larger cases rejected), we test all assignments with other bits fixed. We never use the registered identity to fill unknown bits or stop early; only if all successful decodings across the full search converge to one unique identity do we check if it matches the registered identity.
(3) Meaning of "verification failure... does not imply identity loss": For the exact same post-attack records, the baseline rule fails, whereas the suppression-aware rule succeeds—showing that baseline failure alone does not prove the underlying identity information in these observations is irrecoverably lost.
Response to W1
W1. Fairness of Experimental Comparisons
We thank the reviewer for the suggestions. All baselines follow their original training and verification protocols and are evaluated under identical model transformations.
(1) RF Query Budget and Threshold. RF uses 1,000 French Alpaca queries, a Z-test (p<10−3), and five independent decisions. SimPrint uses 192 semantic queries for RS-based identity recovery and exact matching.
We acknowledge that RF performs comparably to SimPrint under most transformations except model merging. We also agree with the reviewer that, ideally, comparisons should align the query budget, false positive rate, or utility degradation. Given the differences in verification mechanisms and query budgets, we additionally provide a unified model utility evaluation to enable a fairer assessment of performance trade-offs.
(2) Model Utility. Changes in average downstream accuracy after embedding (percentage points, pp):
SimPrint incurs only 1.64 pp average utility loss, below CTCC (6.65), SF (9.03), and iSeal (12.06). Although IF and RF incur smaller losses, SimPrint additionally enables exact identity recovery under model merging (Table 1), offering a favorable utility–robustness trade-off.
(3) SF Clean FSR. SF's 10% Clean FSR on Qwen3-8B is the measured reproduction result under the original configurations, without model-specific tuning. We will clarify this evaluation scope and provide reproducibility details below.
(4) FSR, FPR, and Aggregation. Table 1 reports FSR, not FPR. Per-fingerprint trigger rates measure individual activations, while model-level FSR depends on aggregation rules; FPR requires applying the same acceptance rules to unsigned models. SimPrint instead uses RS decoding and exact identity matching, enabling owner identification. We clarify each baseline's aggregation rules, verification thresholds, and statistical units below and will document them in the revision.
W1.1. Novelty of Contribution 1
We thank the reviewer for highlighting SF and Chain-and-Hash. We acknowledge that ownership verification using partially preserved fingerprints is not new and will revise Contribution 1 accordingly.
SimPrint's novelty lies in an error-correctable semantic identity channel combining structured identity encoding and selective fact editing under black-box access:
(1) Representation. Unlike SF's aggregation of fingerprint triggers, SimPrint encodes identity into an error-correcting codeword distributed across semantic responses.
(2) Embedding. Existing factual responses and a secret mapping encode some identity bits directly; only mismatched positions are edited, reducing utility interference.
(3) Recovery. Unlike threshold aggregation, RS decoding reconstructs complete identities from partially corrupted or erased responses.
Chain-and-Hash uses cryptographic hashing to bind prompts and responses, whereas SimPrint emphasizes identity recoverability under model transformations. Its contribution is thus the integration of structured identity representation, selective embedding, and error-correcting recovery, not merely replacing voting with RS decoding.
(4) Multi-Owner Verification. We embedded four identities into the same base model using 192 shared probes and identical configurations, evaluating all 16 cross-verification pairs (T1).
All 4 correct claims were accepted and all 12 incorrect claims rejected, with no observed cross-owner false acceptance. This demonstrates specific identity discrimination rather than fingerprint-presence detection.
We will revise Contribution 1 to emphasize selective embedding and robust recovery of structured semantic identities.
W1.2. Model Types Used in Experiments
The main experimental table includes both Base and Instruct models. Specifically, Llama-3-8B uses the Instruct checkpoint, while the other models use Base checkpoints. We will provide complete checkpoint names and model types in the revision to avoid ambiguity.