acceptodds
Under review as a conference paper at ICLR 2027

Matryoshka Fingerprinting for Lineage Verification in Large Language Models

Abstract

Protecting the intellectual property of open-source large language models (LLMs) is increasingly important, since post-hoc modifications can obscure whether a deployed model derives from a protected one. However, existing fingerprinting methods are tied to a single model instance and deployment configuration, so their effectiveness degrades under model heterogeneity (hidden-size variation), vocabulary heterogeneity (vocabulary variation), and deployment heterogeneity (template/temperature variation). We propose Matryoshka Fingerprinting (MF), a transferable fingerprinting framework that verifies model lineage across these three forms of heterogeneity in a black-box manner. By freezing all model parameters and jointly optimizing a nested Matryoshka fingerprint embedding across multiple protected models and deployment contexts, MF lets a single fingerprint be reused across models with different hidden sizes and vocabularies, as well as across different templates and decoding temperatures. Extensive experiments show that MF outperforms existing fingerprinting methods, i.e., REEF, ZeroPrint, and PlugAE, in effectiveness, robustness, efficiency, stealthiness, and harmlessness, while extending potentially to cross-family lineage verification and content attribution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, Model Fingerprint, Intellectual Property Protection, Cross-model Lineage, Content Attribution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HdF1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Matryoshka fingerprinting, a fingerprinting scheme to simultaneously protect authorship on multiple models. The scheme achieves this by optimizing a latent fingerprint in the representation space of multiple models simultaneously, with the larger model's representation encapsulating the smaller model's latent a la Matryoshka embeddings. Once the optimized latent is obtained, an unused token is assigned this embedding for each model. The paper evaluates this scheme on multiple model families, as well as multiple deployment settings, demonstrating near perfect fingerprinting rates.

Critical strengths
  1. The black box nature of the fingerprinting scheme is nice.
  2. Thorough investigation on FPR, robustness, stealthiness etc. Also the adaptive attacks presented are nice
  3. Most baselines are covered well.
Critical weaknesses
  1. The experimental results seem to be quite misleading to me. For Matryoshka FP the fingerprint is explicitly optimized for all models, which is not the case for REEF or PlugAE, which only optimize the FP on a single model and are testing its "generalization". In the case of the reported evals, REEF should also not really trigger on other models, since it has a high specificity. A fairer comparison would be to check Matryoshka FP on other related models which were not included for optimizing the FP, or to extend the baselines to use multiple models. A better baseline would be to modify, e.g. PlugAE to account for different models in searching the fingerprint. For example Table 12 reports that using 2 models gives some generalization for Matryoshka FP, and I believe that similar generalization gains would occur for PlugAE as well. In fact this idea of using multiple models for optimizing the fingerprint is not particularly novel, with MergePrint, or RAFP both using this and reporting some gains.

1.1 Further, the metrics of similarity and TPR are not comparable. The paper acknowledges this but still presents them together. I would argue that with similarity one needs to use a threshold to have a low false positive rate and then use that threshold to compute the TPR.

  1. The threat model is unclear. What is the motivation to protect all models in a family with the same fingerprint? Indeed, this is bad from a security perspective since leaking one fingerprint destroys fingerprints across the whole model family (Nasery et al 2025). I get the generalization arguments but these seem to be insufficiently explored in the paper empirically, e.g. the empirical results from from Table 12 cast some doubt on the efficacy of MF, especially under varying hidden sizes.
Questions for authors
  1. Could the authors provide intuition on the differences between their method and PlugAE, which lead to much better performance, e.g. on the Mistral models in Table 1.
  2. What is the motivation of using the same fingerprint to protect all models in a family?

The following are minor comments and questions

  1. How is the vocab manipulation attack done?
  2. What is the threshold for detection and the number of fingerprints?
  3. What is Δm\Delta_m in Prop 1? It would be good to have some actual numbers for this, e.g. it seems that Hm∼5.5H_m \sim 5.5 for δ=0.1\delta=0.1 and vocab size of 32k.
  4. In terms of adaptive attacks, the attacks (suppress-top-k, suppress-top-word) from Nasery et al 2026 seem to be directly applicable here. Do the authors have a sense of how these would work?
  5. All experiments seem to be for invisible tokens, so why can an attacker not just strip those out?
  6. In Table 8 what does the perplexity value denote and how was it computed?

refs Are robust llm fingerprints adversarially robust?, Nasery et al, SaTML'26

1 comment
AuthorsOpenReview ↗

Thank you Reviewer HdF1. We answer each question below.

R1 (Why MF performs better than PlugAE). Both MF and PlugAE optimise token embeddings to encourage models to produce a target response. However, the key difference is how the fingerprints are constructed. PlugAE generates fingerprints for individual models, while MF jointly optimises a shared fingerprint across multiple models. This allows MF to account for differences between models during optimisation, rather than relying on a fingerprint designed for one model to transfer to others. This helps explain MF's better performance in Table 1, particularly on Mistral models. Moreover, MF supports models with different hidden sizes, whereas PlugAE's fingerprint embeddings have a fixed dimensionality. Another important advantage of MF is its stronger robustness against model modifications. To verify this, we have conducted additional experiments comparing MF with PlugAE under fine-tuning and pruning on four models: Llama-2-7B, Mistral-7B-v0.1, Qwen2.5-7B, and Phi-3-mini-4k-instruct. The results are summarised below.

  • Fine-tuning robustness. MF consistently outperforms PlugAE across all four models under 200, 400, and 800 fine-tuning steps. MF maintains detection rates of 87.4–100%, compared with only 38.6–72.43% for PlugAE.
  • Pruning robustness. MF also consistently outperforms PlugAE under pruning ratios of 10%, 15%, and 20%. MF maintains detection rates of 91.6–100%, compared with 43.74–89.72% for PlugAE.

R2 (Why use the same fingerprint for multiple models). Model providers often release many versions and sizes of their models. If every model needs a different fingerprint, the provider must create, store, and verify each one separately. MF makes this easier by allowing multiple models to share the same fingerprint. This reduces the work needed to manage fingerprints and makes ownership verification simpler across different models.

R3 (How is the vocab manipulation attack done). We have described how this attack is performed in Appendix E.1.2 on Page 12 (Paragraph 3).

R4 (Detection threshold and number of fingerprints). In our experiments, we measure TRR directly by checking whether each fingerprint query produces the expected response. Therefore, no detection threshold is needed to determine whether an individual query succeeds. In reality, when verifying model ownership using multiple fingerprint queries, the detection threshold can be set based on the application's requirements. Moreover, the number of fingerprint queries used in each experiment is provided in Appendix E.1.

R5 (Clarification of Proposition 1). In Proposition 1, Δm\Delta_m is the minimum logit gap between the target token and its strongest competing token across the considered prompt templates and generation positions. Proposition 1 shows that when this gap is large enough, the target response remains stable across different templates, even when the decoding temperature changes within the specified range. This explains why MF can consistently trigger the same response under different prompt templates. We will clarify the definition of Δm\Delta_m and add a numerical example to make the result easier to understand.

R6 (More adaptive attacks). Following your suggestion, we have conducted additional experiments to evaluate MF against three adaptive attacks based on Nasery et al.: SuppressNeighbor, SuppressTop-k, and SuppressLookahead. We have tested each attack using 100 fingerprint queries on two models, Llama-2-7B and Mistral-7B-v0.1. The results are summarised below.

  • SuppressNeighbor: MF achieves detection rates of 96% and 100%, respectively.
  • SuppressTop-k: MF achieves detection rates of 82% and 100%, respectively.
  • SuppressLookahead: MF achieves detection rates of 79% and 100%, respectively.

R7 (Why not simply remove invisible tokens). As discussed in Appendix G, MF supports both invisible and visible tokens, each with its own trade-offs. Invisible tokens are harder to notice but can be removed by text sanitization. Visible tokens are more noticeable but cannot be removed by simply stripping invisible characters. Users can choose the token type that best suits their needs, as explained in Appendix G.

R8 (What does perplexity mean in Table 8). Perplexity (PPL) is a well-known metric that is used to measure how likely a piece of text is under a language model. Lower PPL generally means that the text looks more natural to that model. As described in Appendix E.1.2, we use 100 clean prompts and four reference models (OPT-6.7B, Mistral-7B-v0.1, Qwen2.5-7B, and Phi-3-mini). We calculate PPL using the average negative log probability of the tokens. We compare clean prompts with prompts containing fingerprints, both before and after word swapping, random deletion, and random addition. Table 8 reports the average PPL. We will explain this calculation more clearly in the paper.

Reviewer_1Z8COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Matryoshka Fingerprinting (MF), a method for black-box LLM lineage verification. It jointly optimizes a nested fingerprint embedding across heterogeneous models and deployment settings, assigns it to fingerprint tokens before release, and verifies suspect models through predefined responses.

Critical strengths
  • The paper studies an important and interesting problem.
  • The method supports a shared fingerprint across models with different hidden sizes and vocabularies, simplifying verification of multiple protected models.
Critical weaknesses
  • The evidence for transfer to unseen heterogeneous derivatives remains insufficient.
  • Prior work is mischaracterized, and relevant baselines are missing.
Questions for authors
  1. Table 1 mainly reports models included in joint optimization, while Table 12 shows that some models with different hidden sizes achieve high TRR only after being included in optimization. Since some of these models are not true descendants of the protected model, these results do not clearly show MF’s transferability to unseen heterogeneous derivatives. The authors should further evaluate descendants with clear derivation paths that are excluded from optimization and have different hidden sizes or vocabularies.

  2. The Introduction states that “However, these existing fingerprints are tied to a single model instance and a fixed deployment configuration, leaving them ineffective once the model- or deployment-related settings change” (Annepaka & Pakray, 2025). However, some previously mentioned methods, such as SRAF (Wang et al., 2025a), already consider jointly optimizing fingerprints across a base model and its homologous variants. In addition, the experiments do not compare against other model provenance methods, such as ErrorTrace [1] and LLMmap [2].

[1] ErrorTrace: A Black-Box Traceability Mechanism Based on Model Family Error Space [2] LLMmap: Fingerprinting for Large Language Models

1 comment
AuthorsOpenReview ↗

Thank you Reviewer 1Z8C. We address each point below.

R1 (Transfer to unseen models). Table 1 shows that MF can protect models with different hidden sizes and vocabularies using a shared fingerprint when they are included in joint optimisation. Table 12 instead tests unseen models, where TRR drops to 0% in some cases (green/red rows). The key distinction is whether these models share a true training lineage, rather than just a family name. As Reviewer 2K1a points out, a 14B model is not necessarily derived from a 7B model, even if both belong to the Qwen series. Unfortunately, model providers do not always disclose their full training histories, so we cannot tell which models belong to the same "model family". As explained in Appendix G, for this reason, in Tables 1 and 12, we included both models with documented derivation paths and independently trained models that merely share a family name to provide a broader evaluation rather than limiting the results to favourable cases. Together, these results show three key findings:

  1. Joint protection: MF can jointly protect heterogeneous models, even if they do not share a training lineage, as long as they are included in optimisation (see Table 1 and Table 14).
  2. Transferability: MF can transfer to unseen derivatives with documented training lineage, as demonstrated by the Llama-2-7B and Mistral-7B families in Table 12.
  3. Transfer limitations: Transfer is not guaranteed for models that merely share a family name without a documented derivation path, as shown by the green/red rows in Table 12.

We will make these distinctions clearer in the revision.

R2 (Comparison with more baselines). We have added experiments comparing MF with two additional baselines, SRAF and ErrorTrace. Regarding LLMmap, as discussed in Section 4.1, our existing baseline, ZeroPrint, has already been shown to outperform LLMmap in prior work. Therefore, we focus on adding comparisons with SRAF and ErrorTrace rather than repeating previously established results. The new experimental findings are summarised below.

  • Cross-model transferability. Both baselines show limited transferability across models. SRAF achieves a detection rate of 99–100% on the original Qwen2.5-7B but drops to 32–53% on Qwen2.5-7B-Instruct and 0–8% on other Qwen-family models. ErrorTrace also shows inconsistent performance, with detection rates ranging from 1.8% to 92.2% across Llama-2 variants and from 9.9% to 77.7% across Mistral variants. In addition, ErrorTrace produces high false positive rates on some unrelated models, reaching 49.5% on Llama-2-7B and 45.5% on Phi-3-mini in the corresponding experiments.

  • Prompt-template robustness. Both baselines are sensitive to changes in prompt templates. SRAF achieves detection rates of 62.5–85.0% across different prompt templates on Qwen2.5-7B. ErrorTrace achieves 43.3–100% across the tested models and templates.

  • Temperature robustness. Both baselines are sensitive to changes in decoding temperature. For SRAF, the detection rate drops from 100% at temperature 0.2 to 57.5–62.5% at temperature 1.0. ErrorTrace also shows performance variations.

These additional experiments took considerable time, and we have only just completed them. Fortunately, we managed to finish before the discussion deadline. We sincerely apologise for sharing these results at the last minute and appreciate the reviewer's patience and understanding.

Reviewer_2K1aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Matryoshka Fingerprinting (MF) to verify LLM lineage across models with different hidden dimensions, vocabularies, chat templates, and decoding temperatures. Keeping the backbone weights frozen, MF jointly optimizes a shared continuous embedding Z∈Rk×DZ \in \mathbb{R}^{k \times D} across multiple models by slicing the prefix Z[:,1:dm]Z[:, 1:d_m] to match each model's hidden dimension dmd_m, combined with template randomization and a margin loss. Before releasing the models, the optimized vectors are written into each model's vocabulary/embedding table (either by adding invisible Unicode tokens or overwriting rare tokens). Lineage is verified via black-box API queries by checking the Target Response Rate (TRR).

Critical strengths
  • Clever adaptation of Matryoshka representations: Using prefix slicing Z[:,1:dm]Z[:, 1:d_m] to share gradients across models with different hidden sizes is a neat engineering trick to bind multiple checkpoints with a single continuous optimization run.

  • Strong temperature and template stability: Combining context-randomized injection with a logit margin loss (Lmar\mathcal{L}^{mar}) effectively prevents the target sequence probability from diffusing at higher temperatures (T=1.0T=1.0), outperforming PlugAE.

  • Comprehensive attack evaluations: The paper tests a wide range of post-hoc modifications (QLoRA, Wanda pruning, feature scaling, input paraphrasing, and t-SNE outlier checks).

Critical weaknesses
  • Misleading claim of handling "model heterogeneity" vs. actual lineage transfer: The paper claims MF solves hidden-size heterogeneity (e.g., between Qwen2.5-7B at 3584, Qwen1.5-7B at 4096, and Qwen3-14B at 5120). However, looking closely at Table 1 vs. Table 12, MF achieves 100% TRR across different hidden sizes in Table 1 only because all those models were explicitly included in the joint optimization. When a model with a different hidden size is held out as an unseen derivative (Table 12, green/red rows), TRR is literally 0.0%. Moreover, a 14B model is not a post-hoc derivative of a 7B model—they are independently pre-trained checkpoints. Jointly implanting the same backdoor token into MM known models before release is multi-model watermarking, not zero-shot lineage verification across hidden sizes.

  • "Freezing all model parameters" is technically inaccurate: The abstract and intro emphasize that MF "freezes all model parameters". Yet Step 21 of Algorithm 1 requires inserting Z∗Z^* into or overwriting rows of each protected model's input token embedding matrix WEW_E before release. Since WEW_E is part of the model parameters, MF is an invasive/proactive weight-modification method on the embedding layer (unlike truly passive black-box methods like ZeroPrint or TRAP). This distinction should be stated more transparently.

  • Contradiction in the LLM paraphrasing defense (Appendix E.1.2): Under Input Manipulation, the authors state that an external LLM (Qwen2.5-0.5B-Instruct) rewrites the query at temperature 0.7, which "typically destroys or discards the invisible fingerprint tokens", yet Table 2 reports 100.00% TRR across all four models after paraphrasing. If the trigger tokens τ\tau are stripped or regenerated by the paraphraser, how does the target model still output the abnormal answer ("Friday" instead of "Tuesday") on a clean semantic question? Even if visible tokens were used, a generative paraphraser at T=0.7T=0.7 will not copy arbitrary out-of-context trigger words verbatim 100% of the time.

  • Unrealistic Content Attribution extension (Appendix F.2): The proposed "Blue table" mechanism inserts an invisible fingerprint token before every space whenever the KGW green-token count is weak. This is essentially fragile text steganography, not LLM watermarking—a simple regex or Unicode normalization strips all invisible characters immediately, and it does not test the actual generative distribution of the LLM. I suggest toning down or removing this claim from the abstract/main text.

Questions for authors

see weakness.

1 comment
AuthorsOpenReview ↗

Thank you Reviewer 2K1a. We clarify each point below.

R1 (Model heterogeneity and transferability). Table 1 shows that MF can protect models with different hidden sizes and vocabularies using a shared fingerprint when they are included in joint optimisation. Table 12 instead tests unseen models, where TRR drops to 0% in some cases (green/red rows). The key distinction is whether these models share a true training lineage, rather than just a family name. As you point out, a 14B model is not necessarily derived from a 7B model, even if both belong to the Qwen series. Unfortunately, model providers do not always disclose their full training histories, so we cannot tell which models belong to the same "model family". As explained in Appendix G, for this reason, in Tables 1 and 12, we included both models with documented derivation paths and independently trained models that merely share a family name to provide a broader evaluation rather than limiting the results to favourable cases. Together, these results show three key findings:

  1. Joint protection: MF can jointly protect heterogeneous models, even if they do not share a training lineage, as long as they are included in optimisation.
  2. Transferability: MF can transfer to unseen derivatives with documented training lineage, as demonstrated by the Llama-2-7B and Mistral-7B families.
  3. Transfer limitations: Transfer is not guaranteed for models that merely share a family name without a documented derivation path, as shown by the green/red rows in Table 12.

We will make these distinctions clearer in the revision.

R2 (Freezing model parameters). We agree that "freezing all model parameters" is technically inaccurate if token embeddings are counted as model parameters. To clarify, MF keeps all pretrained model parameters frozen during fingerprint optimisation. When implanting the fingerprint, we only modify the selected token embeddings, while all other parameters remain unchanged. We will revise the wording to make this distinction clear.

R3 (Paraphrasing attack). As stated in Appendix E.1.2, Table 2 does not use invisible fingerprint tokens. Instead, we use meaningful words as fingerprint tokens in this experiment. Unlike invisible tokens, meaningful words can remain in the text after paraphrasing. This explains why the fingerprint can still work after the paraphrasing attack.

R4 (Content attribution). We agree that invisible tokens can be removed by text cleaning. However, MF is not limited to invisible tokens and can also use visible words, as discussed in Appendix G. Our red-green-blue method is an additional application of MF, designed to provide extra evidence when the usual green-token signal is weak. We do not claim that it can survive every type of text modification. We will make its purpose and limitations clearer and avoid overstating its robustness.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A big thank you!

Hi all,

We just want to take this opportunity to say a big thank you for all your hard work in organising the conference. With AI developing so quickly, we can imagine how challenging it must be to organise an event of this scale. We really appreciate all the time and effort you have put into making it happen. We hope everything is going well for you all and look forward to meeting you!

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.