acceptodds
Under review as a conference paper at ICLR 2027

From Textual Outputs to Probabilistic Signatures: Robust Black-Box Model Ownership Verification

Abstract

The substantial computational resources required to train large language models (LLMs) make them valuable intellectual property, raising concerns about model theft and unauthorized deployment. Fingerprinting provides a practical approach to ownership verification by determining whether a suspicious model is derived from a protected source model. However, existing black-box methods often fail to provide reliable verification, as textual signals are unstable under stochastic sampling. This issue is pronounced for reasoning models, where long-form generation amplifies the variability introduced by sampling. Although sampled texts vary, the underlying probability patterns remain relatively stable. Motivated by this insight, this work proposes Token-Level Probability Fingerprinting (), a robust framework that leverages probability distributions as more stable verification signals than textual outputs. Since derived models tend to preserve probability patterns similar to their sources, extracts discriminative features by evaluating the token-level probabilities assigned by the source model to responses generated by a suspicious model. Extensive experiments on both reasoning models and standard LLMs demonstrate that achieves effective verification performance approaching white-box methods. Code is available in the supplementary material.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Model Ownership Verification, Model Fingerprinting, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_EA4jOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper addresses black-box model ownership verification and proposes TOPF to overcome the unreliability of textual fingerprints under stochastic sampling. TOPF extracts features from the token-level log-probabilities assigned by the source and reference models to responses generated by a suspicious model. It combines relative similarity to these models with similarity to the source model to determine model provenance. Experiments show that TOPF outperforms existing black-box methods in many settings and achieves performance comparable to white-box methods.

Critical strengths
  • The paper addresses an important limitation of existing methods in handling output variability caused by stochastic sampling. Addressing this issue is practically relevant, particularly for reasoning models, whose long reasoning trajectories may amplify such variability.
  • The proposed method enables ownership verification without modifying the source model. It does not require embedding additional fingerprints into the source model, offering the practical benefit of avoiding potential performance degradation.
Critical weaknesses
  • The risk of false positives for similar models is insufficiently evaluated. Independently fine-tuned models sharing a pre-trained model represent a realistic and challenging scenario for ownership verification. However, the paper does not cover this scenario. This is particularly concerning for TOPF, which uses relative similarity to the source and reference models for verification. Such fine-tuned models may be more similar to the source model than reference models from different model families, potentially leading to false positives. Evaluating false positives on such negatives is therefore necessary to substantiate TOPF’s effectiveness.
  • The substantial performance degradation under model merging is insufficiently analyzed. Table 12 reports an AUC of 0.114 for TOPF under model merging while MET achieves a high AUC of 0.957. The explanation that merging weakens source-specific probability patterns does not adequately explain MET’s strong performance or TOPF’s performance substantially below random guessing. Given that an existing method maintains reasonable performance across the evaluated strategies, including model merging, it is unclear what practical advantage TOPF offers despite its severe failure in this setting. A more thorough analysis is therefore needed to support the claimed benefits of TOPF.
Questions for authors
  • Could the authors clarify which models are used as negatives in the ownership verification evaluation? Do these include models from the same family as the source model? If the negatives come only from different model families, the evaluation may underestimate the risk of false positives.
  • Could the authors also explain how the metrics in Table 1 are aggregated across the different post-development strategies?
8 comments
AuthorsOpenReview ↗

Response 1 to Reviewer `EA4j`

Q1 & W1: Which models are used as negatives? Do they include models from the same family as the source?

In the original evaluation, negatives are models associated with the other source lineages, including their post-development variants. For example, DS-R1-8B is evaluated against Qwen3-8B and GPT-OSS-20B variants. Appendix F lists the checkpoints.

We additionally evaluate two derived models and five related negatives per source, all sharing Llama-3.1-8B Base ancestry. Related negatives originate from other branches of the shared ancestor. DS-R1-8B and Llama-3.1-8B-Instruct belong to separate branches and serve as negatives for each other. TOPF achieves AUC and pAUC of 1.000 on both sources, separating the tested derivatives from models with shared pretraining ancestry.

Protected sourceDerived modelsRelated negativesAUCpAUC
DS-R1-8Bhuihui abliterated; Enkrypt alignedLlama-Instruct; mlabonne abliterated; Fireplace2; Hermes-3; Tulu-31.0001.000
Llama-3.1-8B-Instructmlabonne abliterated; Fireplace2DS-R1-8B; huihui abliterated; Enkrypt aligned; Hermes-3; Tulu-31.0001.000

Table 1. Verification with two derived models and five related negatives per source, all sharing Llama-3.1-8B Base ancestry.

Q2: How are the metrics in Table 1 aggregated across different post-development strategies?

The metrics in manuscript Table 1 are computed from pooled model-level scores for each protected source. Each evaluated checkpoint or deployment variant contributes one verification score and one label.

We combine entries from all evaluated strategies and construct a single ROC curve per source. AUC, pAUC, and TPR@3% are computed from this curve. MD is computed from the corresponding positive and negative score distributions.

Each model entry receives equal weight, so strategies with more evaluated variants contribute more entries to the pooled results. Manuscript Table 12 provides the complementary breakdown by strategy.

W2: Why does TOPF perform substantially below random guessing on the DS merge, while MET performs well? What practical advantage does TOPF offer?

Merging can weaken source-specific probability patterns. In the original DS merging evaluation, TOPF achieves AUC 0.114 on DeepLlama-Chimera-NSFW-8B, while MET achieves 0.957. Its subsequent unavailability prevents further diagnosis of the difference. Results for other protected sources show that this failure does not extend to every evaluated merge.

Protected sourceMerged checkpointsAUCpAUC
Qwen3-8BCavesOfQwen3-8b1.0001.000
Llama-3.1-8B-InstructHalfAbliterated-TIES, ExtraMix0.9030.744
Qwen2.5-7B-InstructStockQwen-2.5-7B, QevaCoT-7B-Stock1.0001.000
Mistral-7B-Instruct-v0.3demi-merge0.9690.679

Table 2. Merge detection for other protected sources, using their respective evaluation pools.

We additionally construct 12 DS merges using linear merging, TIES, and DARE-TIES. Each method combines DS with Llama-Instruct at DS coefficients of 0.25, 0.50, and 0.75, and with Hermes-3 at 0.50. TOPF achieves AUC 0.981 and pAUC 0.915, supporting effective detection across these controlled merges. These results demonstrate TOPF's effectiveness with a small query budget across the tested merges.

MethodAUCpAUC
MET0.6670.667
TOPF0.9810.915

Table 3. DS merging experiment with 12 positives and nine negatives. TOPF uses 20 responses; MET uses 25 prompts with five generations each, totaling 125 responses.


Thank you again for the helpful feedback and thoughtful guidance. We hope our detailed responses successfully resolve your concerns. If there are any remaining uncertainties, we would be happy to clarify them.

AuthorsOpenReview ↗

A Kind Reminder Regarding Our Rebuttal

Dear Reviewer EA4j,

Thank you again for your valuable comments and feedback on our submission. With three days remaining in the discussion period, we would like to kindly follow up and ask whether our rebuttal has sufficiently addressed your concerns.

Should any questions or concerns remain, we would be grateful for the opportunity to provide further clarification. If you find that our responses have resolved the issues you raised, we would sincerely appreciate your consideration of updating your evaluation scores as appropriate.

Thank you very much for your time and consideration.

Best regards,

The Authors

Thank you very much for your detailed response and additional experiments. These results address some of my concerns. However, I would appreciate further clarification regarding the potential for false positives.

My main concern is that negative models independently fine-tuned from the same base model as the protected source may receive high verification scores due to their similarity to the source model, potentially leading to false positives in practical ownership verification.

Although the additional experiments achieve an AUC of 1.000, this does not necessarily rule out false positives under the original decision threshold. In practical ownership verification, the ground-truth labels of suspicious models are unknown, making it infeasible to recalibrate the threshold using the models being verified. Therefore, it is important to evaluate whether the related negative models can be correctly rejected using the threshold calibrated on the original validation set.

I would therefore appreciate clarification on the following points:

  • Were the same reference models as those in the original manuscript used in the additional experiments? Also, were any of the negative models in the additional experiments used during encoder training?
  • Could the authors provide the verification scores of the positive and negative models in the additional experiments?
  • When applying the verification thresholds calibrated on the original validation sets, without recalibrating them for the additional experiments, what are the resulting TPR, FPR, and accuracy?

If the negative models sharing the same base model receive sufficiently low verification scores and the original decision thresholds yield few or no false positives, my concern would be substantially alleviated.

Reviewer_grM3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TOPF, a black-box ownership verification method for LLMs that replaces unstable textual signals with token-level probabilities: responses generated by a suspicious model are scored under the source model and a few open reference models, the resulting probability matrix is encoded by a small contrastively trained CNN-Transformer, and ownership is decided by whether the embedding is relatively closer to the source center than to the reference centers. Experiments on three reasoning models and three instruction-tuned LLMs, with real community-released derivatives (adapters, fine-tuning, merging, quantization) and parameter-free deployments (system prompts, RAG, input paraphrasing), show AUC/TPR@3% approaching white-box baselines on reasoning models and robustness to sampling temperature. Ablations cover reference-set size, probe domain, query budget, and per-perturbation performance.

Critical strengths
  1. The idea is straight-forward and easy to follow.
  2. Using the source model’s token-level log-probabilities on the suspect’s responses is a simple, well-motivated signal.
  3. Derivatives are real checkpoints from the community rather than author-constructed fine-tuned ones.
Critical weaknesses
  1. The contribution of the learned encoder is not isolated. The plain NLL baseline already matches or exceeds TOPF in AUC on reasoning models (0.965 vs 0.946 on DS-R1-8B), and the obvious intermediate baseline, the relative softmax of Eq. (6) computed on mean log-probabilities under source and references, without any trained encoder, is never reported, even though Table 11 computes essentially this quantity. NLL is also omitted for standard LLMs. Without this comparison it is unclear whether the CNN-Transformer adds anything beyond reference normalization.
  2. The negative set is small and structurally narrow (the other two families per setting), model merging essentially fails (AUC 0.114 in Table 12), and no adaptive attack is evaluated despite the threat model allowing arbitrary post-processing. The notion of “derived” is also left ambiguous: DS-R1-Distill-Llama-8B is itself a fine-tune of Llama-3.1-8B, yet the paper does not say how TOPF treats such cross-setting lineage.
Questions for authors
  1. Would TOPF flag Llama-3.1-8B-Instruct as derived from DS-R1-8B (or vice versa)? How is “derived” defined when the source itself is a fine-tune of another model?
  2. Why does model merging fail, and does the failure persist with more or different reference models?
  3. How does TOPF behave when the suspect’s outputs are paraphrased by another model, or under temperatures above 1.5?
5 comments
AuthorsOpenReview ↗

Response 1 to Reviewer `grM3`

Q1 & W2: Would TOPF flag Llama-3.1-8B-Instruct as derived from DS-R1-8B, or vice versa? How is “derived” defined when the source itself is a fine-tuned model?

Based on the Hugging Face Model Tree, we define a derived model as a descendant of the protected source checkpoint. This definition also applies when the protected source is itself fine-tuned. Models on other branches of a shared ancestor are treated as related negatives.

Protected sourceDerived modelsRelated negativesAUCpAUC
DS-R1-8Bhuihui abliterated; Enkrypt alignedLlama-Instruct; mlabonne abliterated; Fireplace2; Hermes-3; Tulu-31.0001.000
Llama-3.1-8B-Instructmlabonne abliterated; Fireplace2DS-R1-8B; huihui abliterated; Enkrypt aligned; Hermes-3; Tulu-31.0001.000

Table 1. Verification with two derived models and five related negatives per source, all sharing Llama-3.1-8B Base ancestry.

DS-R1-8B and Llama-3.1-8B-Instruct originate from separate branches of Llama-3.1-8B Base, so each is a related negative for the other. We additionally evaluate two derived models and five related negatives per source. TOPF achieves AUC and pAUC of 1.000 on both sources, separating the tested derivatives from models with shared pretraining ancestry.

Q2 & W2: Why does model merging fail, and does the failure persist with more or different reference models?

Merging can alter the probability patterns inherited from a source, potentially weakening its fingerprint. This is a plausible explanation for the original DS failure on DeepLlama-Chimera-NSFW-8B, which achieved AUC 0.114. Its checkpoint became inaccessible during the rebuttal period, preventing further diagnosis. Results for the other protected sources show that this failure does not extend to every evaluated merge.

Protected sourceMerged checkpointsAUCpAUC
Qwen3-8BCavesOfQwen3-8b1.0001.000
Llama-3.1-8B-InstructHalfAbliterated-TIES, ExtraMix0.9030.744
Qwen2.5-7B-InstructStockQwen-2.5-7B, QevaCoT-7B-Stock1.0001.000
Mistral-7B-Instruct-v0.3demi-merge0.9690.679

Table 2. Merge detection for other protected sources, using their respective evaluation pools.

We additionally construct 12 DS merges using linear merging, TIES, and DARE-TIES. Each method combines DS with Llama-Instruct at DS coefficients of 0.25, 0.50, and 0.75, and with Hermes-3 at 0.50. TOPF achieves AUC 0.981 and pAUC 0.915, demonstrating strong detection across these merges. TOPF exceeds MET's ranking in this experiment, and these results extend the evidence for merging robustness.

MethodAUCpAUC
MET0.6670.667
TOPF0.9810.915

Table 3. DS merging experiment with 12 positives and 9 negatives. TOPF uses 20 responses; MET uses 25 prompts with five generations each, totaling 125 responses.

AuthorsOpenReview ↗

Response 2 to Reviewer `grM3`

Q3: How does TOPF behave when the suspect’s outputs are paraphrased by another model, or under temperatures above 1.5?

We additionally evaluate output paraphrasing and summarization using Qwen2.5-3B-Instruct. The rewriting model receives the query and response without detector feedback, and both detectors remain fixed. TOPF outperforms LLMmap under both transformations. These results demonstrate TOPF's robustness to the tested output post-processing transformations.

Output transformationMethodDS AUCDS pAUCLlama AUCLlama pAUC
ParaphrasingLLMmap0.7780.7440.6110.487
ParaphrasingTOPF1.0001.0001.0001.000
SummarizationLLMmap0.6670.4870.3890.487
SummarizationTOPF1.0001.0000.8890.744

Table 4. Output post-processing with two positives and nine negatives per source.

We also evaluate temperature 1.6 on an expanded DS pool containing 11 positives and 30 negatives, including five related negatives. TOPF retains AUC 0.958, compared with 0.973 at temperature 0.6, while LLMmap decreases from 0.821 to 0.506. These results support TOPF's robustness to increased sampling temperature in this setting.

MethodAUC at T = 0.6AUC at T = 1.6
LLMmap0.8210.506
TOPF0.9730.958

Table 5. Temperature stress test on DS-R1-8B.

W1: Does the learned encoder improve verification beyond NLL and reference normalization?

We agree that NLL achieves higher overall AUC on DS in the original evaluation, while TOPF performs better at low FPR. To isolate the encoder's contribution, we additionally evaluate three baselines using the same responses and probability traces.

  1. Relative likelihood averages token log probabilities within each answer and then equally across answers. It applies softmax with scale 1 across the source and reference means and uses the source probability.
  2. Maximum-reference margin subtracts the highest reference mean log probability from the source mean.
  3. Pooled classifier extracts the mean, standard deviation, and 10th/50th/90th percentiles from each channel, yielding 20 features. A linear classifier is trained on the original encoder training split, and its answer-level source probabilities are averaged.
MethodDS AUCDS pAUCLlama AUCLlama pAUC
Relative likelihood0.6970.7670.7270.782
Maximum-reference margin0.6610.7510.6620.772
Pooled classifier0.8090.8140.8190.782
TOPF0.9730.9530.9070.886

Table 6. Comparison with probability-based baselines. DS uses 11 positives and 30 negatives; Llama uses 9 positives and 24 negatives. Each pool includes five related negatives.

TOPF achieves the highest AUC and pAUC among these methods on both sources. These comparisons support the contribution of learned sequence representations beyond likelihood normalization and pooled statistics.


Thank you again for the helpful feedback and thoughtful guidance. We hope our detailed responses successfully resolve your concerns. If there are any remaining uncertainties, we would be happy to clarify them.

AuthorsOpenReview ↗

A Kind Reminder Regarding Our Rebuttal

Dear Reviewer grM3,

Thank you again for your valuable comments and feedback on our submission. With three days remaining in the discussion period, we would like to kindly follow up and ask whether our rebuttal has sufficiently addressed your concerns.

Should any questions or concerns remain, we would be grateful for the opportunity to provide further clarification. If you find that our responses have resolved the issues you raised, we would sincerely appreciate your consideration of updating your evaluation scores as appropriate.

Thank you very much for your time and consideration.

Best regards,

The Authors

Reviewer_C2VMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TOPF uses text responses to determine whether a model was derived from a protected language model. The verifier scores these responses with the source model and public reference models, then uses a CNN-Transformer trained with supervised contrastive learning to encode the probability traces. The paper evaluates the method on reasoning and instruction-tuned models under several deployment and parameter changes.

Critical strengths
  1. The setting is useful: the model being tested only needs to return text, while the verifier computes probabilities locally. This makes the approach relevant to services that do not expose token probabilities.
  2. The low-FPR results on reasoning models are encouraging. The paper also includes architecture ablations, reference-model comparisons, cost measurements, and results for individual transformations. These experiments help show where the method works and where it struggles.
Critical weaknesses
  1. Several parts of the evaluation need to be clarified before the main conclusions can be adequately assessed. The first issue is what counts as a descendant. The benchmark assigns labels using equality of the configured pretrained_model field (benchmark/benchmark.py, lines 405-414). Table 16 groups Qwen2.5-Math-7B and Qwen2.5-Coder-7B-Instruct under Qwen2.5-7B-Instruct, but does not establish that they were derived from that particular checkpoint. This matters for ownership verification: a model from the same family could otherwise be attributed to the wrong checkpoint. The evaluation needs documented ancestry and negative examples that share an ancestor with the source without being its descendants. The comparison with white-box methods also needs a common test set. The default benchmark adds deployment variants only for black-box methods (benchmark.py, lines 54-100). Several of these variants are easy for most methods in Table 12. Until results are reported on the same checkpoints, it is difficult to judge the claim that TOPF approaches white-box performance. There is a further mismatch in threshold selection. Table 2 describes a 70/30 validation/test split, a midpoint-of-means threshold, and 20 trials. The supplied evaluation path instead selects a Youden threshold from the current score set (benchmark.py, lines 691-692 and 1040-1049). This does not establish that Table 2 used test data to select thresholds, but the code does not explain how that table was produced. The authors should provide the corresponding script, model-level splits, validation thresholds, and raw test scores. The feature definition is also unclear. Equation (1) conditions probabilities on the query, but the supplied inference path scores the response alone (instruct_model.py, lines 125-134; moco.py, lines 870-916 and 1054-1058). It also truncates different tokenizers' traces to the same token count, which does not necessarily align the same text (moco.py, lines 1090-1095). The authors need to explain which representation was actually used. Missing encoder weights, model centers, probe data, and a complete training and center-generation procedure make these differences harder to resolve.
  2. The results leave substantial gaps in the robustness claim and in the evidence for the proposed design. The merging result is particularly concerning. Merging is included in the threat model, yet TOPF has an AUC of 0.114 on DS-R1-8B in Table 12, compared with 0.957 for MET. Table 15 lists only one merged checkpoint for this source, so this result has limited coverage. Even so, it is a clear weakness in a setting the paper claims to address. The aggregate AUC of 0.919 over parameter changes does not resolve it. The robustness claim should be narrowed or supported by a more thorough merging evaluation. Similarly, paraphrasing inputs does not establish robustness to post-processing outputs. The baseline comparison is incomplete. NLL has higher overall AUC than TOPF on DS-R1-8B (0.965 versus 0.946), while TOPF performs better at low FPR. TOPF also has a small AUC advantage on Qwen3-8B (0.961 versus 0.957). The low-FPR improvement is worth studying, but NLL is missing from Table 1(b), and both NLL and TRAP are missing from Table 2. TRAP is the strongest method on Llama3.1-8B in Table 1(b), so its omission from the threshold-based evaluation is especially relevant. Table 8 compares architectures, but does not show which part of TOPF produces the gain. Simpler relative-likelihood scores and pooled probability features should be tested under the same conditions. Keeping the embeddings fixed while changing the final score would also help separate the contribution of the representation from the scoring rule. Finally, one reference outperforms three on DS-R1-8B and Qwen3-8B in Tables 3 and 13. The claim of insensitivity to reference choice needs to account for these results.
Questions for authors
  1. What evidence establishes the checkpoint ancestry used for the positive labels? How does TOPF perform on models that share an ancestor with the source but are not its descendants?
  2. Which script produced Table 2, and how were validation thresholds kept separate from test data? Please provide the model-level splits and raw scores. Can the white-box comparison also be reported on a common checkpoint set?
  3. Were the reported probability traces conditioned on the query? How were tokens aligned across tokenizers, and where are the training procedure and assets needed to reproduce these features?
  4. Do the low-FPR gains hold against simpler relative-likelihood and pooled-feature baselines? Why were NLL and TRAP omitted from the comparisons noted above?
  5. How should the robustness claim be interpreted given the merging result? Is there evidence for robustness to output post-processing, beyond the input-paraphrasing experiments?
7 comments
AuthorsOpenReview ↗

Response 1 to Reviewer `C2VM`

Q1: What evidence establishes the checkpoint ancestry used for the positive labels? How does TOPF perform on models that share an ancestor with the source but are not its descendants?

A model is positive when its parameter lineage passes through the protected source checkpoint. Related negatives originate from other branches of a shared ancestor. Ancestry evidence comes from the Hugging Face Model Tree and the authors' model cards, supplemented by adapter configurations and merge recipes where available. The pretrained_model field records this assignment.

Protected sourceDerived modelsRelated negativesAUCpAUC
DS-R1-8Bhuihui abliterated; Enkrypt alignedLlama-Instruct; mlabonne abliterated; Fireplace2; Hermes-3; Tulu-31.0001.000
Llama-3.1-8B-Instructmlabonne abliterated; Fireplace2DS-R1-8B; huihui abliterated; Enkrypt aligned; Hermes-3; Tulu-31.0001.000

Table 1. Verification with two derived models and five related negatives per source, all sharing Llama-3.1-8B Base ancestry.

We additionally evaluate two derived models and five related negatives for each source. The related negatives come from other branches of the shared Llama-3.1-8B Base checkpoint. TOPF achieves AUC and pAUC of 1.000 for both sources, showing that it distinguishes the tested source derivatives from models with shared pretraining ancestry.

Q2: Which script produced Table 2, and how were validation thresholds kept separate from test data? Please provide the model-level splits and raw scores. Can the white-box comparison also be reported on a common checkpoint set?

The Table 2 script was not included in the supplementary package. We will provide it with the model-level splits and raw scores in the revision. In our evaluation, we randomly assign approximately 70% of the positive and negative models to validation and the remaining 30% to testing. For each source and method, we set the threshold to the midpoint of the positive and negative validation-score means:

θ=mean⁡(Sval+)+mean⁡(Sval−)2.\theta = \frac{\operatorname{mean}(S^{+}_{\mathrm{val}}) + \operatorname{mean}(S^{-}_{\mathrm{val}})}{2}.

The threshold is fixed before evaluating the test models, and we report the average test accuracy over 20 random splits. Further details of this protocol are provided in Section 5.2.

Regarding white-box evaluation, these methods require access to model parameters. Deployment variants such as system prompts and RAG preserve the source model's parameters. Their weight identity with the source is directly observable under white-box access, making separate weight-based detection unnecessary. We include these variants to evaluate whether ownership remains detectable from their modified responses under black-box access.

Q3: Were the reported probability traces conditioned on the query? How were tokens aligned across tokenizers, and where are the training procedure and assets needed to reproduce these features?

We would like to clarify that the probability traces used in our experiments are conditioned on the query, as defined in Equation (1). The source and reference models compute response-token probabilities given the query and preceding response tokens. The supplementary code reflects an earlier version, which we will update to the latest version.

Each source or reference model uses its own tokenizer, and its probability trace is processed by a separate convolutional extractor. We truncate the traces to a common length for feature fusion. Training and verification use the same feature construction, and ownership is determined by the learned representation's similarity to the source relative to the references.

Following Section 4.2 and Appendices A and B, we train the encoder with supervised contrastive learning and average the training embeddings to construct model centers. We will provide the encoder weights and model centers in the revision.

AuthorsOpenReview ↗

Response 2 to Reviewer `C2VM`

Q4: Do the low-FPR gains hold against simpler relative-likelihood and pooled-feature baselines? Why were NLL and TRAP omitted from the comparisons noted above?

MethodDS AUCDS pAUCLlama AUCLlama pAUC
Relative likelihood0.6970.7670.7270.782
Maximum-reference margin0.6610.7510.6620.772
Pooled classifier0.8090.8140.8190.782
TOPF0.9730.9530.9070.886

Table 2. Comparison with probability-based baselines. DS uses 11 positives and 30 negatives; Llama uses 9 positives and 24 negatives. Each pool includes five related negatives.

We additionally evaluate three baselines using the same responses and probability traces:

  1. Relative likelihood averages token log probabilities within each answer and then equally across answers. It applies softmax with scale 1 across the source and reference means and uses the source probability.
  2. Maximum-reference margin subtracts the highest reference mean log probability from the source mean.
  3. Pooled classifier extracts the mean, standard deviation, and 10th/50th/90th percentiles from each channel, yielding 20 features. A linear classifier is trained on the original encoder training split, and its answer-level source probabilities are averaged.

TOPF achieves the highest AUC and pAUC on both sources, supporting the benefit of learned sequence representations beyond likelihood normalization and pooled statistics.

As explained in Section 5.2, TRAP's GCG optimization failed to converge on the evaluated LRMs. NLL was omitted from the standard-LLM comparison because of its weak low-FPR performance.

Q5: How should the robustness claim be interpreted given the merging result? Is there evidence for robustness to output post-processing, beyond the input-paraphrasing experiments?

Merging can alter the probability patterns inherited from a source, potentially weakening its fingerprint. This is a plausible explanation for the original DS failure on DeepLlama-Chimera-NSFW-8B, which achieved AUC 0.114. Although this checkpoint became inaccessible during the rebuttal period, we broadened the merging evaluation. Results for the other protected sources are summarized below.

Protected sourceMerged checkpointsAUCpAUC
Qwen3-8BCavesOfQwen3-8b1.0001.000
Llama-3.1-8B-InstructHalfAbliterated-TIES, ExtraMix0.9030.744
Qwen2.5-7B-InstructStockQwen-2.5-7B, QevaCoT-7B-Stock1.0001.000
Mistral-7B-Instruct-v0.3demi-merge0.9690.679

Table 3. Merge detection for other protected sources, using their respective evaluation pools.

MethodAUCpAUC
MET0.6670.667
TOPF0.9810.915

Table 4. DS merging experiment with 12 positives and 9 negatives. TOPF uses 20 responses; MET uses 25 prompts with 5 generations each, totaling 125 responses.

We additionally construct 12 DS merges using linear merging, TIES, and DARE-TIES. We combine DS with Llama-Instruct at DS mixture coefficients of 0.25, 0.50, and 0.75, and with Hermes-3 at 0.50. TOPF achieves AUC 0.981 and pAUC 0.915, demonstrating strong detection across these merges. TOPF exceeds MET's ranking in this experiment. These results extend the evidence for merging robustness.

Output transformationMethodDS AUCDS pAUCLlama AUCLlama pAUC
ParaphrasingLLMmap0.7780.7440.6110.487
ParaphrasingTOPF1.0001.0001.0001.000
SummarizationLLMmap0.6670.4870.3890.487
SummarizationTOPF1.0001.0000.8890.744

Table 5. Output post-processing robustness.

We also evaluate output paraphrasing and summarization with Qwen2.5-3B-Instruct. The rewriting model receives the query and response. TOPF outperforms LLMmap under both transformations. It preserves separation after paraphrasing, while summarization reduces performance on Llama. These results support robustness to the post-processing procedures.

AuthorsOpenReview ↗

Response 3 to Reviewer `C2VM`

W1: What counts as a descendant, and why were Qwen2.5-Math-7B and Qwen2.5-Coder-7B-Instruct assigned to Qwen2.5-7B-Instruct?

We thank the reviewer for identifying this inconsistency. The models used in our submitted experiments were HuatuoGPT-o1-7B and Bespoke-Stratos-7B, both derived from Qwen2.5-7B-Instruct. Table 16 retained the model list from an earlier version. We will correct these entries in the revision.

W2: One reference outperforms three on DS-R1-8B and Qwen3-8B. How does this support insensitivity to reference choice?

We agree that one reference outperforms three on DS-R1-8B and Qwen3-8B, as shown in Table 3. Table 9 further examines whether performance remains stable when changing the composition of three references. Replacing Gemma with MiMo changes AUC from 0.946 to 0.949 on DS-R1-8B and from 0.961 to 0.959 on Qwen3-8B. These small changes support robustness to the tested reference substitution when using multiple reference models.

W2: Separate the contribution of the representation from the scoring rule.

Scoring ruleDS AUCDS pAUCLlama AUCLlama pAUC
Cosine similarity0.9730.9530.8940.886
TOPF0.9730.9530.9070.886

Table 6. Effect of the scoring rule with fixed representations.

We compare TOPF's full scoring rule with cosine similarity to the source center while keeping the encoder and embeddings fixed. TOPF's scoring rule improves Llama AUC, while DS results and pAUC on both sources remain unchanged. This isolates the scoring contribution from the learned representation.


Thank you again for the helpful feedback and thoughtful guidance. We hope our detailed responses successfully resolve your concerns. If there are any remaining uncertainties, we would be happy to clarify them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.