acceptodds
Under review as a conference paper at ICLR 2027

From Bits to Beliefs: Recovering Owner Identity for Black-Box Verification of Large Language Models

Abstract

Open-weight large language models (LLMs) can be copied, modified, and redeployed behind black-box APIs, making post-release ownership verification difficult. Existing black-box fingerprinting methods largely treat verification as a detection problem, asking whether registered query-response behaviors remain observable in a suspect model. Such evidence can degrade under fine-tuning, pruning, quantization, model merging, and serving-time prompt changes. We propose SimPrint, a recoverable semantic fingerprinting framework that instead formulates ownership verification as identity recovery from noisy black-box behavior. SimPrint encodes a private owner identity into an error-correcting codeword and distributes its bits across natural binary question-answering probes. It selectively implants only probes that must deviate from the base model through a low-interference batch update, and later parses suspect-model responses into bits or erasures to recover the registered identity. Because verification requires only input-output queries, SimPrint remains applicable when model weights and activations are inaccessible. Experiments on three open-weight LLMs show reliable identity recovery in both clean and modified settings, robustness to fine-tuning, pruning, quantization, model merging, and serving-time perturbations, and comparable downstream utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Model Fingerprinting, Ownership Verification, Black-box Verification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kYZ6OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SimPrint, a method to fingerprint language models using multiple questions and answers. The method first fixes a code-word derived from an error correcting code. Then it maps each bit of the code-word to binary questions, whose answers are determined (mostly) by the code-word bit. The question-answers are then injected into the LLM using a knowledge editing method inspired by AlphaEdit. The paper demonstrates the effectiveness of the scheme on three models, as well as the robustness to fine-tuning, merging, pruning and quantization. The method also leads to negligible drop in the utility of the model.

Critical strengths
  1. The method proposed is very nice. I like the idea of using coding theory to come up with code words that can survive some amount of degradation or corruption induced by model modifications. This seems like a very natural idea.
  2. The paper is quite well-written and the method is easy to follow.
  3. I liked the False positive rate analysis, though I would have liked to see some empirical evidence as well.
Critical weaknesses

My main issue with the paper is the empirical evaluation, especially wrt prior work

  1. While I like the idea of using multiple queries for establishing ownership, Semantic Watermarking (Gloaguen et al 2026) has similar flavour as well. Indeed, from Table 1, it seems that RF has very similar performance to the proposed method except under model merging. However it is unclear how many queries and what p-value threshold was used for this method --- ideally one needs to match either number of queries (i.e. 192), false positive rate or utility degradation among different methods. Similarly for Scalable Fingerprinting, the clean fingerprinting rate itself is very low (10% on Qwen3). This indicates that the hyper-parameters were not tuned sufficiently for this method, since the clean rate should be 100% for a well-tuned baseline. It is also unclear to me how the FPR for SF. IF or CTCC are computed --- are the fingerprints treated independently, or is a model said to be fingerprinted if greater than k% of fingerprints fire?

1.1 Scalable fingerprinting (Nasery et al 2025) provides an approach to answer the question from lines 63-65 of ownership from partially recoverable model behaviour. Even chain-and-hash (Russonivich&Salem 2025) achieves a similar goal successfully. Hence, I believe that Contribution 1 overclaims being the first work to address this question.

1.2 It is unclear if the experiments only involve base models or instruct models as well.

  1. For results under attack, particularly serving time perturbations, output perturbations (Nasery et al 2026) do not seem to be considered. I belieive that the attacks (SuppressTopK, SuppressNeighbours) would achieve high ASR against the present scheme. In fact there are no adaptive attacks on the system at all. While this is not crucial, it would strengthen the paper to know what the practical bottlenecks of the scheme are.

Refs LLM Fingerprinting via Semantically Conditioned Watermarks, Gloaguen et al, ICLR 2026 Scalable Fingerprinting of LLMs, Nasery et al, NeurIPS 2025 Hey, that's my model! Introducing Chain-and-Hash, Salem & Russonivich, ICLR 2026 Are robust llm fingerprints adversarially robust?, Nasery et al, SaTML 2026

Questions for authors
  1. Does the method survive output pertubations (e.g. Suppress-top-k from Nasery et al 2026)?
  2. Can the experimental details for the baselines (e.g. learning rates, epochs for backdoor based methods, number of queries/p-value threshold for RF), as well as the other evaluation settings be reported?
4 comments
AuthorsOpenReview ↗

Response to W1

W1. Fairness of Experimental Comparisons

We thank the reviewer for the suggestions. All baselines follow their original training and verification protocols and are evaluated under identical model transformations.

(1) RF Query Budget and Threshold. RF uses 1,000 French Alpaca queries, a Z-test (p<10−3p<10^{-3}), and five independent decisions. SimPrint uses 192 semantic queries for RS-based identity recovery and exact matching.

We acknowledge that RF performs comparably to SimPrint under most transformations except model merging. We also agree with the reviewer that, ideally, comparisons should align the query budget, false positive rate, or utility degradation. Given the differences in verification mechanisms and query budgets, we additionally provide a unified model utility evaluation to enable a fairer assessment of performance trade-offs.

(2) Model Utility. Changes in average downstream accuracy after embedding (percentage points, pp):

MethodLlama3.2-3BLlama3-8BQwen3-8BAverage
IF+0.04−0.04−0.07−0.03
RF−0.68−1.07+0.22−0.51
CTCC−3.11−11.46−5.37−6.65
SF−8.66−17.56−0.87−9.03
iSeal−15.20−9.96−11.04−12.06
SimPrint−1.40−1.69−1.82−1.64

SimPrint incurs only 1.64 pp average utility loss, below CTCC (6.65), SF (9.03), and iSeal (12.06). Although IF and RF incur smaller losses, SimPrint additionally enables exact identity recovery under model merging (Table 1), offering a favorable utility–robustness trade-off.

(3) SF Clean FSR. SF's 10% Clean FSR on Qwen3-8B is the measured reproduction result under the original configurations, without model-specific tuning. We will clarify this evaluation scope and provide reproducibility details below.

(4) FSR, FPR, and Aggregation. Table 1 reports FSR, not FPR. Per-fingerprint trigger rates measure individual activations, while model-level FSR depends on aggregation rules; FPR requires applying the same acceptance rules to unsigned models. SimPrint instead uses RS decoding and exact identity matching, enabling owner identification. We clarify each baseline's aggregation rules, verification thresholds, and statistical units below and will document them in the revision.

W1.1. Novelty of Contribution 1

We thank the reviewer for highlighting SF and Chain-and-Hash. We acknowledge that ownership verification using partially preserved fingerprints is not new and will revise Contribution 1 accordingly.

SimPrint's novelty lies in an error-correctable semantic identity channel combining structured identity encoding and selective fact editing under black-box access:

(1) Representation. Unlike SF's aggregation of fingerprint triggers, SimPrint encodes identity into an error-correcting codeword distributed across semantic responses.

(2) Embedding. Existing factual responses and a secret mapping encode some identity bits directly; only mismatched positions are edited, reducing utility interference.

(3) Recovery. Unlike threshold aggregation, RS decoding reconstructs complete identities from partially corrupted or erased responses.

Chain-and-Hash uses cryptographic hashing to bind prompts and responses, whereas SimPrint emphasizes identity recoverability under model transformations. Its contribution is thus the integration of structured identity representation, selective embedding, and error-correcting recovery, not merely replacing voting with RS decoding.

(4) Multi-Owner Verification. We embedded four identities into the same base model using 192 shared probes and identical configurations, evaluating all 16 cross-verification pairs (T1).

Model / Claimed IdentityOwner001Owner002Owner003Owner004
Owner001✓✗✗✗
Owner002✗✓✗✗
Owner003✗✗✓✗
Owner004✗✗✗✓

All 4 correct claims were accepted and all 12 incorrect claims rejected, with no observed cross-owner false acceptance. This demonstrates specific identity discrimination rather than fingerprint-presence detection.

We will revise Contribution 1 to emphasize selective embedding and robust recovery of structured semantic identities.

W1.2. Model Types Used in Experiments

The main experimental table includes both Base and Instruct models. Specifically, Llama-3-8B uses the Instruct checkpoint, while the other models use Base checkpoints. We will provide complete checkpoint names and model types in the revision to avoid ambiguity.

AuthorsOpenReview ↗

Response to W2 and Q1-2

W2 & Q1. Output Perturbations and Adaptive Attacks

We thank the reviewer for highlighting output perturbations during model serving. We evaluated 16 fixed and 12 confidence-selective attack configurations on two fingerprinted models and explored verification improvements without retraining or additional model queries.

(1) Attack Settings.

We evaluated two attacks on Qwen3.5-9B and Llama-3-8B-Instruct:

  • SuppressTopK: Sets the logits of the top-kk highest-probability tokens to negative infinity at the first nn candidate continuation positions.
  • SuppressNeighbor: Additionally suppresses morphological-prefix neighbors of high-probability tokens using a fixed suppression set at the first nn positions.

With k∈{1,5}k\in\{1,5\} and n∈{1,5}n\in\{1,5\}, these yield 16 fixed configurations.

For confidence-selective attacks, candidate confidence is computed from the original mean negative log-likelihood (NLL):

c=exp⁡(−mean⁡(NLL)).c=\exp(-\operatorname{mean}(\mathrm{NLL})).

Perturbation (k=1,n=1k=1,n=1) is applied only when c≥τc\geq\tau, with τ∈{0.5,0.7,0.9}\tau\in\{0.5,0.7,0.9\}, yielding 12 additional configurations.

(2) Suppression-Aware Verification and List Decoding.

The original SimPrint verifier failed under all 16 fixed configurations. However, hard suppression can disrupt semantic decisions without necessarily destroying the underlying identity information. We therefore explored three strategies requiring no retraining or additional queries:

  1. Erasure Verification: Mark a bit as unknown if either candidate is suppressed, then apply RS erasure decoding.
  2. Suppression-Aware Verification: Retain the original decision when both scores are finite; infer the bit from suppression of the more probable candidate when only one is suppressed; otherwise mark it as unknown.
  3. Suppression-Aware + List Decoding: Enumerate unknown-bit assignments and apply RS(24,8) decoding. Match against registered identities only if all successful decoding outcomes yield a unique identity.

Enumeration is limited to 12 unknown bits (4,096 local decodings); larger cases are rejected. Neither enumeration nor filtering uses the expected identity, and the search never terminates early upon finding a registered identity.

(3) Identity Recovery Results.

The table reports exact identity recovery successes.

AttackOriginalErasureSuppression-AwareSuppression-Aware + List
SuppressTopK0/82/88/88/8
SuppressNeighbor0/80/82/88/8
Confidence-Selective8/1212/1212/1212/12
Clean2/22/22/22/2

Suppression-aware verification recovered all identities under SuppressTopK and confidence-selective attacks; adding list decoding recovered the six remaining SuppressNeighbor failures.

These Neighbor cases contained 1–11 unknown bits, requiring at most 2,048 enumerations, with exactly one distinct identity candidate per exhaustive search. Additionally, no false acceptance was observed in 13 historical clean records from unsigned models (0/13).

These results show that verification failure under output suppression does not necessarily imply identity loss. Suppression-aware extraction and bounded list decoding recovered all identities under the tested attacks without additional model queries. We will clarify the attack interface, decoding rules, and experimental limitations in the revision.

Q2. Baseline Hyperparameters and Reproducibility Details

We summarize baseline configurations and verification protocols, following the original papers or official implementations.

MethodTraining ConfigurationVerification Protocol
IFIF-Adapter: LR 10−210^{-2}, 20 epochs, adapter dim 16, effective batch 48; IF-SFT: LR 2×10−52\times10^{-5}, 3 epochs, effective batch 6410 trigger queries, greedy decoding
SFLR 5×10−55\times10^{-5}, 30 epochs, batch 8, 1,024 training fingerprints128 trigger queries
CTCCLoRA, LR 10−410^{-4}, 12 epochs, rank 8, alpha 16, effective batch 1695 trigger queries; temperature 0.95, top-p 0.7, max tokens 512
RFAdafactor, LR 2×10−52\times10^{-5}, 2,500 steps, batch 64, 250 warmup steps, cosine schedule1,000 French Alpaca queries; Z-test p<10−3p<10^{-3}; 5 independent decisions
iSealLR 10−310^{-3}, 15 epochs, batch 2, 10 training samples10 plaintext query samples

We will provide complete checkpoint names, training configurations, query budgets, aggregation rules, and statistical definitions in the revision to support reproducibility.

I thank the authors for their detailed response. There are a few comments I would like to make ---

RF query budget, model utility

Thanks for the clarification on verification for RF. I also like the idea of iso-utility comparison (with the caveat being how one defines utility, since some methods and settings would be better at certain datasets). However, from the table in the rebuttal, it seems that RF still dominates SimPrint in the utility-fingerprinting trade-off, with model merging being the only robustness gain that SimPrint offers.

SF Clean FSR. SF's 10% Clean FSR on Qwen3-8B

Note that using the same hyper-parameters from the SF paper with a different model (afaict the original SF work did not evaluate on Qwen3 at all) is not a fair comparison. Ideally one needs to tune the hyper-parameters for each method on each model considered.

FSR and aggregation

Sorry, the FPR in my original review was a typo, I did mean to ask about fingerprint success rate. It is still unclear to me how the aggregation works for other baselines, for example for IF is something verified only if all 10 fingerprint responses are correct? The table in the second response does not seem very clear to me.

Adaptive attacks

Thanks for this additional experiment. It appears that SimPrint can be easily bypassed by output perturbations. I am unsure of the mitigations described in the rebuttal, could the authors write them out more clearly? For example, what does "Retain the original decision when both scores are finite; infer the bit from suppression of the more probable candidate when only one is suppressed;" mean? Does one require access to logits for this? The new verification protocols appear to be an extension of the original method and probably require closer scrutiny, hence I would like it if the authors described them as clearly as possible. I also do not understand what is meant by "verification failure under output suppression does not necessarily imply identity loss."

Overall I think my original concerns with the paper still remain and hence I would keep my rating.

We thank Reviewer kYZ6 for the follow-up. We re-verified our logs and verification code, and respond below.

1. RF Query Budget, Utility, and Trade-offs Model merging is where SimPrint's advantage is most pronounced, but it is neither the only robustness gain nor the only practical advantage over RF:

  • Beyond Merging: On Llama-3-8B fine-tuned on Alpaca (Table 1), SimPrint recovers the identity (100%), whereas RF fails all five tests (0%; p=0.507,0.054,0.033,0.045,0.161>10−3p = 0.507, 0.054, 0.033, 0.045, 0.161 > 10^{-3}). On Qwen3-8B (Table 1), RF also drops to 70% / 0% under Random Pruning (0.1 / 0.2) and 80% / 60% under Taylor Pruning (0.1 / 0.2), while SimPrint keeps 100%.

  • Overall Trade-off: Trade-offs should jointly consider transformations, query budgets, and identity recovery. For a 1.131.13 pp average utility difference (−1.64-1.64 vs. −0.51-0.51 pp), SimPrint uses 5.2×5.2\times fewer queries per test (192 probes vs. 1,000 queries), achieves exact 64-bit identity recovery, and maintains 100% robustness where RF degrades.

2. SF's Clean FSR on Qwen3-8B While tuning SF for Qwen3-8B could raise its Clean score, the observed robustness gap does not stem solely from Qwen3-8B:

  • On Llama-3-8B and Llama-3.2-3B, SF reaches 88.2% and 88.6% Clean matching, yet incurs large utility drops (−17.56-17.56 and −8.66-8.66 pp) and still collapses to 0%–4.1% after TIES and DARE+TIES merging (Tables 1 & 8).

  • Thus, merging severely disrupts SF even when initial implantation is strong. We will clarify that 10.45% on Qwen3-8B is the default-setting result rather than SF's tuned upper bound.

3. FSR Definitions and Aggregation For IF, no: it does not require all 10 responses to be correct. Table 1 reports the triggered fraction across the 10 queries (e.g., 8/10 on Llama-3-8B after Alpaca is 80%), without converting partial scores into binary model-level pass/fail decisions.

MethodMetric in Table 1Unit
IFFraction of 10 queries containing target stringPer-query retention
SFFraction of 128 queries exactly matching registered tokensPer-query retention
CTCCFraction of 95 queries containing target markerPer-query retention
RFFraction of 5 tests (1,000 queries/test) passing p<10−3p < 10^{-3}Model-level detection
iSealFraction of 10 queries achieving BLEU≥50\text{BLEU} \ge 50Per-query retention
SimPrintExact match (w^=w\hat{w} = w) of 64-bit identity after RS decodingComplete identity recovery

Thus, IF, SF, and CTCC retain partial per-query credit, whereas SimPrint is evaluated under a stricter criterion requiring complete, exact identity recovery. We will clarify this in Section 4.1.

4. Output Perturbations and Extended Verification

(1) Logit access: Yes. This defense extension requires logits/candidate likelihoods to detect whether preset answers are hard-suppressed. It applies to interfaces exposing scores (e.g., logprobs), not strictly text-only APIs. We will state this explicitly.

(2) How the new rules work: Hard suppression sets top-token probabilities to zero, giving any candidate containing them zero probability. When suppression occurs:

Post-Attack SuppressionBit Reading Rule
Only one candidate suppressedSelect the bit of the suppressed candidate.
Both candidates suppressedMark bit as unknown (?).
  • Why Rule 1 works: Because the attack only suppresses high-probability tokens, a single suppressed candidate must have been the higher-probability choice before perturbation (no pre-attack ranking needed). This rule alone (Suppression-Aware) recovers 8/8 identities under SuppressTopK and 12/12 under Confidence-Selective.
  • Bounded Enumeration (+ List Decoding): For remaining unknown bits (≤12\le 12 allowed, i.e., ≤4,096\le 4,096 local RS(24,8)\text{RS}(24,8) decodings; larger cases rejected), we test all 2u2^u assignments with other bits fixed. We never use the registered identity to fill unknown bits or stop early; only if all successful decodings across the full search converge to one unique identity do we check if it matches the registered identity.

(3) Meaning of "verification failure... does not imply identity loss": For the exact same post-attack records, the baseline rule fails, whereas the suppression-aware rule succeeds—showing that baseline failure alone does not prove the underlying identity information in these observations is irrecoverably lost.

Reviewer_81PSOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a model watermarking scheme that can be verified against a black box target model. Identity verification is based on a set of binary question answers analysed as a error correcting code. Simulation results exhibit a fearsome 100% accuracy on the 3 tested model in all the tested conditions.

Critical strengths
  • clear presentation
  • simple approach
  • adequate model modification coverage in experiments
Critical weaknesses
  • Limited novelty: localized editing and ECC perspective are each well known. According to contributions summary, the novelty is to "not use a verification-time module". Unfortunately the importance of this contribution stays quite unclear: why is it important ? Isn't RSdecode a verification-time module ?
  • Flawed experimental evaluation: only 3, rather dated, models tested. Proposal performance is always 100% : when the method breaks is not shown. Competing baselines have very bad performance, and repetitions appear inconsistent (Table1: IF,Iseal may have 10 repetitions --min delta=10% , SF may have 1000 -- min delta=0.1%) which together lower confidence in the experiments. Method and baselines do not seem to be compared on equal basis, like a comparable verification token cost. The only RS configuration tested has an expansion factor of 3, which seems overpowered given the bit accuracies reported fig 3.
  • no real false positive experiments: the simple analysis case covering if target model is M_0 is quite limited; there are many models in the wild. Ideally, a verifier facing several black boxes should try to identify the target one.
  • Semantic dimension is insufficiently detailed: how are probes constructed ? My belief is that probe selection impacts both editing impact on utility, FPR, and resistance to updates: a probe "two parallel lines never cross T/F?" will likely be shared by many models if expected bit is True, and its editing to False may harm model utility more than a mundane probe. Furthermore, it is still unclear whether questions and target answers are timestamped too -- text suggests that no, but intuition suggests that then the scheme could be abused: a malicious user could pretend to own any model by first registering some key, and only after model release invent some questions/answers couples that fit the model and the key ?
Questions for authors
  • How is binary answer extracted -- often models are more verbal that a single token answer ? This is relevant in adv system prompt setting for instance
  • Can you summarise other baselines verification costs ? And comment on the very low performance of selected competitors on the bench.
  • what is the impact of temperature on bit error rate ?
  • I missed some positioning wrt "Robust and Efficient Watermarking of Large Language Models Using Error Correction Codes" Xiaokun Luan et al. Pets'25
  • some words about the stealthiness of the approach would be good given it is often a desired property
  • (this is a remark, not a question) why call fingerprint something that modifies the model ? I still don't get it. Are there some countries where citizens get their fingertips hot-iron branded before having their fingerprints registered ? I know many papers currently do this in the LLM realm, and don't hold it against this particular paper.
6 comments
AuthorsOpenReview ↗

Response to W1-W2(1)(2)

W1. Novelty and Verification Mechanism

The core contribution of SimPrint is not a new local editing algorithm or error-correcting code. Rather, it is a unified framework that embeds a pre-registered owner identity into a model's semantic behavior through selective factual editing and enables identity recovery from corrupted black-box responses.

Specifically, SimPrint encodes a 64-bit owner identity into a 192-bit Reed–Solomon (RS) codeword and maps it to natural-language factual question-answer pairs. By leveraging the model's existing knowledge, approximately half of the responses already match the target codeword and require no modification. Only the remaining probes are locally edited, thereby unifying low-interference identity embedding and robust identity recovery within a single framework.

To evaluate identity discrimination, we embedded four distinct owner identities into the same base model using the same probe pool and conducted all 16 pairwise verification trials:

Verification ScenarioTrialsRS Identity Verification
Correct owner claim44/4 accepted
Incorrect owner claim120/12 accepted

These results demonstrate that SimPrint can not only detect a fingerprint but also recover and verify a pre-specified owner identity. Unlike detection mechanisms that rely solely on aggregate matching scores, SimPrint provides an explicit mechanism for recovering identity from corrupted responses.

Regarding the verification-time module, we clarify that RS decoding is indeed a verification-stage algorithm. SimPrint's advantage is not the elimination of verification computation, but rather its ability to recover identity through fixed-candidate scoring and deterministic RS decoding, without training an additional discriminative model or accessing the model's internal parameters during verification. We will clarify this distinction in the revised manuscript.

W2. Experimental Adequacy and Fairness

We thank the reviewer for the constructive suggestions regarding model coverage, robustness limits, statistical reliability, verification cost, and error-correction redundancy. We have carefully re-examined the original results and conducted additional experiments on a new model, robustness failure boundaries, verification costs, and RS code-rate sensitivity.

(1) Model Coverage and Generalization

The original manuscript already evaluates Llama3-8B-Instruct, Llama3.2-3B, and Qwen3-8B. We further extend the evaluation to Qwen3.5-9B, using the same pre-registered 64-bit identity and testing both the clean signed model and various model transformations:

CategoryConditionCodeword Bit AccuracyRS Identity Recovery
CleanSigned model96.35%Success
Fine-tuningLoRA 1 epoch97.92%Success
Fine-tuningLoRA 3 epochs98.96%Success
Fine-tuningLoRA 6 epochs98.96%Success
PruningRandom 10%95.31%Success
PruningRandom 20%95.31%Success
PruningTaylor 10%97.40%Success
PruningTaylor 20%96.88%Success
MergingD-TIES 5:597.92%Success
MergingD-TIES 2:897.92%Success
MergingTIES 5:598.44%Success
MergingTIES 2:898.96%Success
QuantizationGPTQ 4-bit97.92%Success
QuantizationGPTQ 8-bit96.35%Success

The exact owner identity was successfully recovered in all 14 conditions, further extending the model coverage of SimPrint.

(2) Robustness Failure Boundaries

We agree that a comprehensive robustness evaluation should report not only successful recovery but also the conditions under which the method fails.

First, the original manuscript already includes an ablation on editing strength. On Llama3-8B, when λ is set to 1, 5, 20, and 50, the bit accuracies are 72.9%, 99.5%, 96.9%, and 90.6%, respectively, with corresponding fingerprint success rates (FSR) of 0%, 100%, 100%, and 0%. These results indicate that both insufficient and excessive editing strength can lead to identity recovery failure.

We further conducted a magnitude-pruning severity sweep on the same signed Llama3-8B model:

Pruning RatioRS Identity Recovery
10%Success
15%Success
20%Success
25%Success
30%Success
35%Success
40%Failure

Identity recovery succeeds at all six pruning levels from 10% to 35%, but fails at 40%. Specifically, 35% pruning produces 8 erroneous bytes, which remains within the guaranteed correction capability of RS(24,8). In contrast, 40% pruning produces 10 erroneous bytes, exceeding its guaranteed correction capacity of 8 unknown byte errors.

This experiment explicitly identifies an empirical failure boundary and demonstrates that SimPrint does not claim unconditional robustness against arbitrarily severe model degradation.

AuthorsOpenReview ↗

W2 (3)-(5)

(3) Independent Repetitions and Statistical Reliability

All baselines use the training and verification protocols of their original papers and official implementations, without reduced budgets or unfavorable settings:

MethodTraining ConfigurationVerification Protocol
IFIF-Adapter: LR 10−210^{-2}, 20 epochs, dim 16, batch 48; IF-SFT: LR 2×10−52\times10^{-5}, 3 epochs, batch 6410 triggers, greedy decoding
SFLR 5×10−55\times10^{-5}, 30 epochs, batch 8, 1,024 fingerprints128 triggers
CTCCLoRA, LR 10−410^{-4}, 12 epochs, rank 8, alpha 16, batch 1695 triggers, temperature 0.95, top-p 0.7, max tokens 512
RFAdafactor, LR 2×10−52\times10^{-5}, 2,500 steps, batch 64, 250 warmup, cosine schedule1,000 French Alpaca queries per detection, Z-test p<10−3p<10^{-3}, acceptance rate over 5 independent decisions
iSealLR 10−310^{-3}, 15 epochs, batch 2, 10 training samples10 plaintext queries

The observed 10% and 0.1% increments do not imply 10 or 1,000 independent repetitions. Differences in fingerprint counts, verification units, denominators, and aggregation rules mean percentage granularity cannot establish repetition counts or unfairness.

Appendix D.6 reports 10 independent signings from the same unsigned Llama3-8B base, with distinct identities, keys, probes, and seeds. Both conditions use the same 10 signed instances:

ConditionExact RecoveryWrong-Key False AcceptanceUnsigned-Model False Acceptance
Clean10/100/100/10
20% magnitude pruning10/100/100/10

All identities were recovered without observed false acceptances, supporting independent repeatability. We will clarify baseline protocols and statistical reporting in the revision.

(4) Verification Cost and Baseline Fairness

Table 1 compares robustness under each method's protocol on identical backbones and transformations, not equal verification costs. We audited per-verification budgets on Llama3-8B-Instruct using documented configurations and execution logs:

MethodVerification OperationsInput TokensGenerated Tokens
SimPrint192 probes1,7183,072
IF10 generation queries1,339240
SF128 generation queries2,176256
CTCC95 generation queries5,4974,600
RF1,000 generation queries29,256169,000
iSeal10 forward predictions3,760—

These are native, not FLOPs-matched, budgets: IF, SF, CTCC, and RF mainly use autoregressive generation; iSeal uses position-wise prediction.

Fairness also requires considering model utility preservation. The following table reports downstream accuracy changes relative to unsigned models (percentage points, pp):

MethodLlama3.2-3BLlama3-8BQwen3-8BMean (pp)
IF+0.04−0.04−0.07−0.03
RF−0.68−1.07+0.22−0.51
CTCC−3.11−11.46−5.37−6.65
SF−8.66−17.56−0.87−9.03
iSeal−15.20−9.96−11.04−12.06
SimPrint−1.40−1.69−1.82−1.64

SimPrint incurs only 1.64 pp average utility loss, substantially less than CTCC, SF, and iSeal. Although IF and RF preserve utility better, SimPrint achieves stronger identity recovery robustness under the evaluated transformations. Thus, its robustness gains do not come at the expense of severe utility degradation.

Together, these results clarify the trade-offs among identity recovery robustness, utility preservation, and native verification cost, beyond a single success-rate comparison.

(5) RS Code Rate and Redundancy

We agree that the threefold expansion of RS(24,8) requires justification. However, bit accuracy alone cannot determine the required redundancy, since RS correction depends on erroneous bytes rather than individual bits.

Under 40% magnitude pruning, 180/192 bits remain correct (93.75%), but the 12 erroneous bits span 10 bytes, exceeding RS(24,8)'s guaranteed capacity of 8 unknown byte errors. Identity recovery therefore fails despite high bit accuracy.

We further conducted a paired effective code-rate ablation across all verification records:

Effective RS ConfigurationIdentity Identification Rate
RS(24,8)100%
RS(20,8)96.4%
RS(16,8)95.8%
RS(12,8)96.1%
RS(10,8)71.4%

RS(24,8) achieves the highest recovery rate, while substantially reducing redundancy, particularly to RS(10,8), markedly degrades performance. The non-monotonic intermediate results further indicate sensitivity to the specific error distribution.

Overall, these analyses clarify SimPrint's empirical robustness limits, independent repeatability, verification costs, and redundancy–recovery trade-offs, providing a more precise characterization of its applicability and limitations.

AuthorsOpenReview ↗

Response to W3-4, Q1-2

W3. False Positives and Anonymous Black-Box Model Identification

We agree that testing false positives only on the original unsigned model M0M_0 is insufficient to assess false ownership claims in realistic settings.

We therefore expanded the unsigned negative controls and conducted ownership identification experiments with multiple anonymous black-box endpoints.

All experiments use the same pre-registered 64-bit identity and RS(24,8) encoding, without adjusting probes, identity claims, or verification rules. The evaluation includes existing and newly held-out unsigned models, plus an anonymous pool of one signed and four unsigned models.

Experimental SettingRS Identity Verification
Existing unsigned test set0/12 false acceptances
Additional held-out unsigned models0/4 false acceptances
Existing signed model1/1 correctly identified
Anonymous mixed pool (1 positive + 4 negatives)Unique correct identification
All-negative pool (4 unsigned endpoints)All rejected

No false acceptances occurred across 16 unsigned trials. The signed model was uniquely identified in the mixed pool, while all unsigned candidates were rejected in the all-negative pool. Beyond the original M0M_0 analysis, these results demonstrate ownership discrimination across heterogeneous unsigned models and anonymous black-box pools.

W4. Semantic Probe Construction, Model Utility, and Pre-registration Security

We thank the reviewer for highlighting probe selection and pre-registration. We clarify three aspects:

First, SimPrint does not rely on arbitrary true/false questions. It uses CounterFact-style fictional entity–relation facts rather than mathematical axioms or universal logical rules. SimPrint preserves responses already matching the target codeword and locally edits only mismatches, minimizing unnecessary parameter updates.

Second, the original manuscript provides evidence on utility and robustness. Table 3 evaluates nine downstream tasks, with average accuracy changes no greater than 0.02 across three models. Tables 1 and 2 further report 100% identity recovery under the evaluated fine-tuning, merging, pruning, quantization, and serving-time input perturbations.

Finally, valid ownership claims require registration information fixed before model release, preventing claimants from selecting probes or modifying mappings after observing model responses.

The Method section and Appendix A.4 already require a pre-release hash commitment through a trusted timestamping service, covering owner identity ww, secret seed κ\kappa, and nonce ss:

c=H(w∥κ∥s).c=H(w\parallel\kappa\parallel s).

Here, κ\kappa controls probe selection and encoding mappings. To explicitly bind the complete probe configuration, we will formalize the commitment in the revision as:

c′=H(w∥κ∥s∥H(P)),c'=H(w\parallel\kappa\parallel s\parallel H(P)),

where PP contains the finalized questions, candidate answers, target answers, and encoding mappings. Verification must confirm both the recovered identity and the probe configuration's consistency with the pre-release commitment and timestamp.

Thus, post-release construction or modification of question-answer pairs cannot support valid ownership claims, as they fail the pre-registration consistency check. This clarifies the security assumptions without altering SimPrint's embedding or RS decoding mechanisms.

Q1. Binary Answer Extraction and Output Format Variations

SimPrint predefines two candidate answers per semantic probe, secretly mapped to bits 0 and 1. A bit is recovered when the selected candidate is unambiguous; refusals, irrelevant responses, or ambiguous answers are marked as erasures for subsequent RS decoding.

We further evaluated input and response format variations on Llama-3-8B-Instruct:

Test ConditionBit AccuracyRS Identity Recovery
Original prompt without chat template96.35%Success
Multi-turn conversation history95.83%Success
Minimum-length answer instruction94.79%Success
Full-sentence rewriting instruction94.79%Success

Despite some bit errors introduced by these variations, RS decoding successfully recovers the complete identity in all tested conditions.

Q2. Baseline Verification Cost and Performance Differences

All baselines follow their original training configurations and official verification protocols, without reduced budgets or unfavorable hyperparameters. Their lower success rates reflect actual performance under the evaluated transformations and native protocols, not artificially weakened settings.

As detailed in W2 (3) and W2 (4), we report training configurations, verification costs, and utility changes to support a comprehensive comparison of robustness, verification overhead, and utility trade-offs.

Reviewer_zh7XOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work addresses the problem of LLM ownership verification under black-box access. The authors point out that existing fingerprinting methods rely on detecting predefined behaviors, which can be vulnerable to model transformations. To address this issue, they propose SIMPRINT, a recoverable black-box fingerprinting framework. SIMPRINT encodes owner identity into natural binary QA probes using error-correcting codes and selectively injects fingerprints to preserve model utility. In experiments, the authors evaluate SIMPRINT on three open-weight LLMs under various model transformations and serving-time perturbations, demonstrating its robustness compared to existing fingerprinting methods while maintaining comparable downstream utility.

Critical strengths
  • The paper addresses an important problem in LLM ownership verification. The ability to verify ownership through black-box APIs makes the proposed approach practically relevant.
  • The proposed method is clearly designed. The use of error-correcting codes for robust verification and selective low-interference injection for utility preservation are both reasonable and well-motivated.
Critical weaknesses

While the experimental results demonstrate that SIMPRINT is more robust than existing methods, its fundamental advantages over prior fingerprinting approaches remain unclear.

  • The authors argue that existing detection-oriented methods are vulnerable to model transformations. However, these methods can also mitigate the loss of individual fingerprints by using threshold-based verification over multiple fingerprint observations. SIMPRINT provides an explicit mechanism for recovering corrupted fingerprints through error-correcting codes, which is a strength. However, RS codes have already been explored in prior work such as iSeal.
  • The contribution of identity recovery itself to robustness is unclear. The authors emphasize formulating ownership verification as identity recovery as a key contribution. However, the ability to recover an owner identity from corrupted responses seems to mainly come from the error-correcting capability of RS codes. Therefore, it is unclear what additional benefits the identity recovery formulation itself provides.
Questions for authors
  • Could the authors explain why SIMPRINT’s fingerprints remain robust to system prompt changes despite their specific natural QA format? Does this robustness also hold under substantially different input formats, response formats, or interaction settings?
  • Do the authors plan to release the implementation code to support reproducibility?
  • Could the authors clarify whether FSR uses comparable evaluation units across SIMPRINT and the baselines in Table 1? SIMPRINT counts verification as successful when the owner identity can be recovered from the full set of 192 probes, even if some individual probes fail. Do the baselines measure success at the level of individual fingerprints or an entire verification procedure? If the evaluation units differ, could the authors explain how this affects the interpretation of the reported comparisons?
3 comments
AuthorsOpenReview ↗

Response to W1-W2

W1. Prior Use of Threshold-Based Verification and RS Error-Correcting Codes

We thank the reviewer for this comment. We agree that threshold-based aggregation can mitigate partial fingerprint failures and that RS codes have been used in prior work, including iSeal. SimPrint's core contribution is not a new error-correcting algorithm, but the construction of stable, low-interference, and recoverable semantic identity fingerprints through selective factual editing.

(1) Robustness stems not only from RS error correction but also from the inherent stability of factual fingerprints. SimPrint encodes a 64-bit owner identity into a 192-bit codeword and maps it to natural-language factual question-answering behaviors. By leveraging the model's existing knowledge, SimPrint selectively edits only approximately half of the responses that are inconsistent with the target encoding. As shown in Table Q3, the per-bit matching accuracy ranges from 94.27% to 100% across all 42 combinations of three backbone models and 14 model conditions, even without RS decoding. This demonstrates that the high fingerprint retention rate after model transformations is not solely attributable to error correction.

(2) Selective editing balances identity recovery and model utility. Table 5 shows that both selective editing and editing all 192 positions achieve successful identity recovery. However, selective editing yields higher average downstream utility (0.59 vs. 0.56), demonstrating that a recoverable identity fingerprint can be established without modifying every factual response.

(3) SimPrint differs from iSeal in its information carrier and embedding mechanism. iSeal applies RS encoding to generation targets to improve tolerance to perturbations in verification responses. In contrast, SimPrint distributes encoded identities across multiple independent factual question-answering behaviors through selective factual editing, targeting robustness against fine-tuning, pruning, quantization, and model merging.

Therefore, SimPrint's contribution lies in combining stable factual behaviors, low-interference selective editing, and recoverable identity encoding, rather than simply applying error-correcting codes to existing fingerprint detection results.

W2. Contributions of Identity Recovery Beyond RS Error Correction

RS codes provide error-correction capability, whereas identity recovery modeling determines what information a fingerprint carries and how it is embedded and verified. SimPrint encodes a pre-registered owner identity into a model's factual behaviors, enabling verification to recover and validate a specific identity rather than merely performing binary classification through aggregated scores.

This distinction is particularly relevant in multi-owner settings. To investigate this, we conducted an additional multi-owner cross-verification experiment (T1). Using the same base model, a shared set of 192 factual probes, and identical fingerprinting configurations, we independently embedded four owner identities and evaluated all 16 cross-verification combinations.

Table T1: Cross-Identity Verification Among Four Owners

Actual Model / Claimed IdentityOwner001Owner002Owner003Owner004
Owner001✓ / ✓✗ / ✓✗ / ✓✗ / ✓
Owner002✗ / ✓✓ / ✓✗ / ✓✗ / ✓
Owner003✗ / ✓✗ / ✓✓ / ✓✗ / ✓
Owner004✗ / ✓✗ / ✓✗ / ✓✓ / ✓

Each cell reports RS-based identity verification / count-based threshold verification, where ✓ indicates acceptance and ✗ indicates rejection. The count-based method uses a fixed threshold requiring at least 148 matching bits out of 192 (≥148/192).

RS-based identity verification correctly accepts all four genuine identity claims and rejects all 12 incorrect claims, whereas the fixed count-based threshold accepts all 16 claims. This demonstrates that the original binary detection threshold, while capable of identifying fingerprinted models, cannot directly distinguish different owners sharing the same probe set.

We emphasize that this does not imply threshold-based methods are inherently incapable of distinguishing multiple owners. In this experiment, recalibrating the threshold can also distinguish all four owners. However, SimPrint's verification rule is determined by the pre-registered encoded identity and does not require recalibrating decision thresholds against other owners' fingerprinted models.

Thus, the value of identity recovery is not additional noise tolerance beyond RS error correction, but rather representing a specific owner identity as information that can be encoded, recovered, and verified, and combining this capability with selective factual editing to establish an explicit multi-owner verification mechanism. We will further clarify this contribution in the revised manuscript.

AuthorsOpenReview ↗

Response to Q1-Q3

Q1. Robustness to System Prompts and Input/Output Format Variations

We thank the reviewer for this question. SimPrint's robustness to prompt and interaction format variations primarily stems from the stability of factual response preferences and the RS error-correction mechanism. Changes in system instructions or response formats do not necessarily alter the model's relative preferences between factual answer candidates. Even when some encoded bits are affected, RS decoding can still recover the identity within its error-correction capability.

To further evaluate this property, we tested identity recovery under different interaction contexts and response formats on Llama-3-8B-Instruct.

Evaluation ConditionBit AccuracyRS Identity Recovery
Original prompt without chat template96.35%Success
Multi-turn conversation history95.83%Success
Concise-answer instruction94.79%Success
Full-sentence restatement instruction94.79%Success

These results demonstrate that SimPrint's verification does not depend on a single prompt format. Even when the interaction context or response requirements change, the factual fingerprints retain sufficient information for successful identity recovery.

Q2. Code Availability and Reproducibility

We thank the reviewer for their interest. We will publicly release the implementation of SimPrint upon acceptance to facilitate the reproduction of our experimental results.

Q3. Fairness of FSR Evaluation Units in Table 1

We thank the reviewer for this suggestion. We further clarify the verification protocols and FSR computation used for different methods in Table 1.

(1) Each method follows its native verification protocol. IF and CTCC primarily measure the proportion of successful trigger instances; SF performs verification using repeated queries and a hit-count threshold; RF aggregates multiple query responses for statistical detection; iSeal determines verification success through response decoding and similarity evaluation; and SimPrint uses the complete set of 192 encoded probes to recover the registered identity.

(2) Table 1 follows fair baseline evaluation settings. We strictly follow the configurations specified in the original papers and official implementations of the baselines, including fingerprint size, query strategy, and verification rules, and evaluate all methods under the same backbone models and model transformation conditions. Each method retains its complete native verification mechanism, avoiding artificial simplification or weakening of the baselines. Therefore, Table 1 reflects the practical robustness of different methods under their standard verification protocols. We will explicitly clarify the FSR evaluation unit for each method in the revised manuscript to improve interpretability.

(3) Additional per-bit metrics further substantiate robustness. Beyond complete identity recovery rates, we additionally report SimPrint's per-bit matching accuracy without RS decoding. Across all 42 combinations of three backbone models and 14 model conditions, the accuracy ranges from 94.27% to 100%. This demonstrates that SimPrint's factual fingerprints exhibit high retention rates after model transformations and that their robustness is not entirely dependent on error correction.

Table Q3: Per-Bit Matching Accuracy Under Different Model Transformations (%)

ConditionLlama3-8BQwen3-8BLlama3.2-3B
Clean100.00100.0098.96
FT Alpaca100.00100.0098.96
FT ShareGPT100.00100.0099.48
FT Dolly99.48100.0098.96
D-TIES 5:598.9696.88100.00
D-TIES 2:899.4896.3598.96
TIES 5:599.4897.40100.00
TIES 2:899.4897.40100.00
Random 0.1100.0095.8398.96
Random 0.298.9695.3197.92
Taylor 0.1100.00100.0099.48
Taylor 0.2100.0094.2799.48
GPTQ 4-bit100.0096.8898.44
GPTQ 8-bit100.0096.3598.96

Table 1 presents the robustness of different methods under their respective official verification protocols, while the additional per-bit results further demonstrate the stability of SimPrint's factual fingerprints independently of RS decoding. This analysis helps avoid conflating different evaluation units and distinguishes the contributions of factual fingerprint stability and RS error correction to final identity recovery.

Thank you for the detailed responses and additional experiments. Most of my concerns have been addressed, and I appreciate the clear and well-motivated design of the proposed method. Therefore, I will raise my score.

However, one concern regarding W2 remains. Similar multi-owner verification might be possible without identity recovery by registering different fingerprint patterns for each owner. Thus, the fundamental advantages of identity recovery over existing approaches still seem somewhat limited. For this reason, I will give a weak accept.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.