acceptodds
Under review as a conference paper at ICLR 2027

Robustly Contestable Ownership Proof from Model Memorization

Abstract

Open-weight models can be adapted and redistributed: Alice trains and releases a model, Bob then fine-tunes it but claims he trained it himself. How can Alice provide a publicly verifiable proof that Bob’s model is derived from hers? We study this goal as a contestable ownership proof: Alice wants to prove that she trained her model and wants to contest any declared lineage that omits her release. Despite progress in model fingerprinting, which embeds detectable signals in models, existing methods do not enable Alice to publicly prove that Bob’s model was derived from hers. In this paper, we present the first such construction: a generic framework that uses cryptography to bind a model fingerprint to its developer’s identity and declared lineage, enabling any third party to verify ownership proofs and lineage challenges. We instantiate the framework with palimpsestic memorization, the tendency of models to memorize later training data more strongly. We implement the resulting protocol as a Python library. To test the robustness of this instantiation, we study adversarial fine-tuning that aims to make the original developer’s ownership claim fail verification. We evaluate six models of different sizes from three families: OLMo, Pythia, and CrystalCoder. For each model, we use linear programming to find a fine-tuning direction predicted to reduce the fingerprint signal as much as possible. Along these tested directions, the ownership test stops detecting the signal only after WikiText perplexity rises substantially.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Model Ownership, Model Provenance, Palimpsestic Memorization, Adversarial Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pkFCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The author proposed a framework to resolve model ownership or lineage. The framework uses palimpsestic memorization characteristics of the model as the fingerprint; combining the fingerprint with cryptographic commitments and append-only public ledgers, the model owner can create a proof (and a timestamp). The model owner can challenge the release of a later-released model to check if this new model is finetuned from previous models without declaring the model lineage. Empirical evaluation is conducted with three families of large language models.

Critical strengths

The structure is clear, and leveraging palimpesestic memorization as a model fingerprint to resolve model ownership is novel. Resolving model ownership is an important field and can lead to practical solutions to protect the properties of the model trainer.

Critical weaknesses

About the structure of the paper:

  1. Palimpsestic memorization is a key concept used in the proposed framework. It shall be defined in the manuscript.

About the guarantee of the proposed mechanism:

  1. What’s the theoretical guarantee of using palimpsestic memorization phenomenon as a fingerprint (i.e., is the fingerprint a 1-to-1 mapping to the model)? For example, if an adversary steals the model, how easy is it to forge a dataset with similar pattern? Also, how likely are two independently trained models to share similar fingerprints? What are the TP, FP, TN, FN for using palimpsestic memorization as a model fingerprint in the contest/challenge?
  2. How is the audit panel selected? How does different selection of audit panels affect the effectiveness of the proof?

Others:

  1. Why focus on LLMs? Does palimpsestic memorization work for other ML models?
Questions for authors

Additional questions:

  1. How is the model initialization verified? Is it randomly initialized or a pretrained model checkpoint from other runs and how to verify them? Proof of Learning-based works also test whether the initial state of the model follows a claimed initialization process (e.g., random init, or a model weights released from another Proof of Learning certificate). Does the new method rely on similar mechanisms to check the model init?
4 comments

We sincerely thank you for your thoughtful review and insightful feedback. We are glad that you find the structure “clear”, and resolving model ownership “an important field”, and our use of palimpsestic memorization to construct model ownership “novel”. We address your concerns and questions below.

Q1 (Model Initialization and Proof of Learning).

We would like to clarify that our protocol does not rely on a Proof-of-Learning initialization certificate: our guarantee concerns subsequent model lineage relative to a base model, so it does not need to verify how that base was trained. The framework accepts both random initialization and pretrained checkpoints as the base model.

W1 (definition and formalization)

We have added an explicit definition of both palimpsestic memorization and memorization in the revised paper Appendix B.3:

  • Palimpsestic memorization: a significant dependence between training order and loss above a fixed threshold.
  • Memorization: a loss gap between used and unused data in training.

W2.1 (Theoretical guarantee)

Thank you for raising this important question. We agree that the formal guarantees of our protocol should be presented more clearly. We’d like to clarify that the fingerprint does not need to be one-to-one. Alice commits to her data before public randomness sets the training order, so Bob cannot forge a matching dataset and order afterwards. Following your suggestion, we give the false-positive and false-negative analysis in the follow-up comment below; we will add it to the revision. We show that a model generated independently of the claimed training schedule is matched to that schedule with bounded small probability. For a second detector that takes a majority vote over panels, we also bound the probability of missing a true match by 0.95% when the target model retains a sufficiently strong loss gap between used and unused data.

W2.2 (audit panel selection)

We agree that the selection of the audit panel is important. The audit panel is sampled pseudorandomly with a cryptographic hash seeded by fresh public randomness. Our guarantees rely on this random sampling. Since the panel is drawn at random from the population, a large enough panel gives a statistic close to the population’s, so different random panels give similar results.

W3 (Why LLMs?)

Thank you for this question. We focus on LLMs because this is where ownership disputes matter most today. Open-weight LLMs are widely shared, fine-tuned and re-released on platforms such as Hugging Face, often without clear lineage declarations. LLMs also give us the experimental setting we need, since open LLMs such as Pythia, OLMo and CrystalCoder publish their training order and intermediate checkpoints. The fingerprint could also work for other model families if training order leaves a detectable signal in their per-example losses.

We hope our clarifications and the added definitions, proofs and experiments address your concerns. If so, we would greatly appreciate it if you would consider raising your score. We are happy to answer any further questions.

Details for W2.1 (false positives and false negatives)

Our fingerprint is not one-to-one, and this is by design. A fine-tuned copy has different weights from the original yet can retain the same training signal. That persistence is precisely what lets the original claimant challenge a later release that omits her claim. Conversely, one model can carry several fingerprints: a model Bob fine-tunes from Alice's can show both his trace and hers (Fig. 1). The fingerprint is evidence of provenance because a genuine descendant can retain the signal, while a model whose losses are independent of the claimant's random schedule matches only with bounded probability.

Can an adversary inspect the model and forge a matching dataset? We assume open weights. Once the honest model is released, the adversary may freely inspect the weights and choose examples by their losses. It must, however, cryptographically commit to the ordered dataset before the public beacon issues the fresh key that determines which entries are used for training and in what order. It can choose the examples, but it cannot then move low-loss examples into favorable positions under that commitment; a new ordering needs a new registration and a fresh key. Forging therefore reduces to passing the test by chance, which is the false-positive event bounded below. Release order is another safeguard: in a formal contest the judge (Def. 1 in App. B.1; Protocol 1) requires the challenger's release record to precede the target's, so a claim registered after the honest model was released cannot be used to contest it.

How likely are two independently trained models to share a fingerprint? Here "independent" means that the target's losses on the registered dataset are independent of the claimant's key. This covers a model trained on the same or overlapping data; what matters is that its training did not depend on which entries the key selected or on their order. The key assigns the training-position labels to the dataset entries uniformly at random, up to negligible cryptographic terms (beacon unpredictability, commitment binding, collisions in the keyed shuffle).

Detector used in our experiments (one panel, permutation threshold; Protocol 2). The audit compares the observed statistic with those from RR random reshufflings of the panel's labels and accepts when pperm≤ap_{\mathrm{perm}}\le a (App. B.3; R=99,999R=99{,}999 and a=0.01a=0.01 in our experiments). Fix the panel. Under independence, the registered labelling of the panel is itself just a random shuffle of the labels, so it is statistically indistinguishable from the RR reshuffled labellings. The observed statistic is therefore equally likely to hold any rank among the R+1R+1 values, and pperm≤ap_{\mathrm{perm}}\le a requires it to rank among the ⌊a(R+1)⌋\lfloor a(R+1)\rfloor largest. Hence the false-positive rate is at most aa per audit and the true-negative rate at least 1−a1-a: 1% and 99% at the level a=0.01a=0.01 used throughout.

Empirical check. We tested whether Pythia's training order is detected in the Pythia-deduped models. These were trained independently, on a deduplicated version of the same corpus (the Pile) and in a different order: nearly the same data, unrelated order.

ModelPermutation p-value
Pythia-1.4B0.00947
Pythia-1.4B-deduped0.14677
Pythia-6.9B0.00031
Pythia-6.9B-deduped0.20655

At the 1% level the original models are detected (true positives) and the independently trained deduped models are not (true negatives). All four models are scored on a shared audit panel of 10,000 Pile sequences, with 99,999 permutations of the Pearson statistic. Two independent models are a sanity check consistent with the bound. Kuditipudi et al. (2025) make the same comparison.

Details for W2.1, part 2: majority detector and false positive

A second detector with a provable false-negative bound. The permutation threshold depends on the data, which makes false negatives harder to bound. App. B.3 announces a second detector with an analytic threshold and a majority vote. App. B.3 now states its learning conditions; we give the definition and analysis below and will add them to the revision. The detector is defined as follows. After the target weights are fixed, the auditor uses fresh public randomness to draw an odd number of independent random subsets (panels) of nn examples from the registered dataset. The number of panels is a public parameter; we use 149 throughout, as in Fig. 6. Each example has a position label uu: zero if unused, otherwise its training position divided by the number of used examples, so u∈[0,1]u\in[0,1]. Let yy be its loss under the target model. On a panel, let CC be the covariance of uu and yy and VV the variance of yy. A panel passes if nC2/V≥τnC^2/V\ge\tau, where τ\tau is a public threshold; a panel with V=0V=0 fails. Since C2/V=Var(u) Corr(u,y)2C^2/V=\mathrm{Var}(u)\,\mathrm{Corr}(u,y)^2, this is the squared Pearson correlation of Protocol 2, scaled by n Var(u)n\,\mathrm{Var}(u) and compared with a fixed threshold instead of a permutation threshold. The audit declares a match if at least 75 of the 149 panels pass.

False positives and true negatives (majority detector). Whatever the nn label values on a panel are, each way of matching them to the panel's examples is equally likely: the matching is a uniform shuffle π\pi. Write uπ(i)u_{\pi(i)} for the label that example ii receives. Standardize the panel's losses as zi=(yi−yˉ)/Vz_i=(y_i-\bar y)/\sqrt V, where yˉ\bar y is their mean on the panel, so that ∑izi=0\sum_i z_i=0 and ∑izi2=n\sum_i z_i^2=n. Then S=∑iziuπ(i)S=\sum_i z_i u_{\pi(i)} has mean zero and satisfies nC2/V=S2/nnC^2/V=S^2/n, so the panel passes only if ∣S∣≥nτ|S|\ge\sqrt{n\tau}.

Reveal the shuffled labels one at a time. Revealing position jj changes the expected final sum by (zj−zˉ>j)(uπ(j)−μj)(z_j-\bar z_{>j})(u_{\pi(j)}-\mu_j), where zˉ>j\bar z_{>j} is the mean of the later losses and μj\mu_j is the mean of the labels not yet revealed: a larger label here leaves a smaller average for the remaining positions. Since labels lie in [0,1][0,1], this change ranges over an interval of length at most dj=∣zj−zˉ>j∣d_j=|z_j-\bar z_{>j}|. Moreover ∑jdj2≤2n\sum_j d_j^2\le 2n: removing zjz_j from the m≥2m\ge2 losses not yet passed lowers their sum of squared deviations by m−1mdj2≥dj2/2\tfrac{m-1}{m}d_j^2\ge d_j^2/2, and the initial sum is nn. The Azuma–Hoeffding inequality for this martingale gives Pr⁡(one panel passes by chance)≤2exp⁡ ⁣(−2nτ2n)=2e−τ.\Pr(\text{one panel passes by chance}) \le 2\exp\!\left(-\frac{2n\tau}{2n}\right)=2e^{-\tau}. At τ=ln⁡100\tau=\ln 100, the threshold used in Fig. 6, this is 2% per panel, so the expected number of passing panels is at most 149(0.02)=2.98149(0.02)=2.98. A match needs 75, so by Markov's inequality its probability is at most 2.98/75<4%2.98/75<4\%. Markov uses only the per-panel probability, so the panels may share one key. Thus, for one audit of an independent target, the false-positive rate is at most 4% and the true-negative rate at least 96%.

This bound is conservative. Because Var(u)≤1/4\mathrm{Var}(u)\le1/4, a panel passes only if ∣Corr(u,y)∣n≥2τ≈4.3|\mathrm{Corr}(u,y)|\sqrt n\ge2\sqrt\tau\approx4.3, while a shuffled correlation has standard deviation 1/n−11/\sqrt{n-1}. For general τ\tau the same argument gives 4e−τ4e^{-\tau}.

Details for W2.1, part 3: false negatives and repeated attempts

False negatives and true positives (majority detector). Fix a genuine target, either the released model or a modified descendant. Assume (i) used examples have lower mean loss than unused ones by at least Δ>0\Delta>0, with variance at most σ2\sigma^2 in each group, and (ii) among used examples, later training position does not correlate positively with loss. With α\alpha the fraction of examples used, the covariance over the dataset splits into a within-used term, nonpositive by (ii), and a between-group term of magnitude at least α(1−α)Δ/2\alpha(1-\alpha)\Delta/2, since used labels average at least 1/21/2. Dividing by the standard deviation of yy, the normalized covariance has magnitude at least κ=α(1−α)Δ2σ2+α(1−α)Δ2,\kappa=\frac{\alpha(1-\alpha)\Delta}{2\sqrt{\sigma^2+\alpha(1-\alpha)\Delta^2}}, because this ratio only grows if the true gap exceeds Δ\Delta. If nκ2≥128n\kappa^2\ge128, a random panel passes with probability at least 5/85/8. A panel can miss the signal in two ways. Its covariance can fall below half the dataset covariance in magnitude: the panel covariance has mean squared error at most 4 Var(y)/n4\,\mathrm{Var}(y)/n because u∈[0,1]u\in[0,1], so Chebyshev's inequality bounds this by 16/(nκ2)≤1/816/(n\kappa^2)\le1/8. Or its loss variance can exceed four times the dataset variance: its expectation is at most the dataset variance, so Markov bounds this by 1/41/4. Outside both events nC2/V≥nκ2/16≥8>ln⁡100nC^2/V\ge n\kappa^2/16\ge8>\ln100, and the panel passes. For a fixed model and key the 149 panels are independent, so Hoeffding's inequality bounds majority rejection by exp⁡(−2⋅149⋅(1/8)2)=e−149/32≈0.95%\exp(-2\cdot149\cdot(1/8)^2)=e^{-149/32}\approx0.95\%; 149 is the smallest odd panel count for which this is below 1%. Thus the false-negative rate is at most 0.95% and the true-positive rate at least 99.05% for genuine targets that satisfy (i)–(ii) with nκ2≥128n\kappa^2\ge128. This condition is sufficient, not necessary.

Measured false negatives (new Fig. 6). Fig. 6 reports the majority detector (149 panels, τ=ln⁡100\tau=\ln100) on the tested checkpoints with their true training order: 1,000 audits per cell, each resampling panels from a fixed pool of scored examples. The false-negative rate moves from 100% to 0% as nn crosses a model-dependent size. It is 0% at n = 3K for Pythia-6.9B and 12B, at 30K for CrystalCoder (100K pool) and for Pythia-1.4B on a 1M pool; for Pythia-1.4B and 2.8B on 10K pools the n = 10K cell is a single decision, which accepts. For OLMo-7B it is 9.8% at 60K and 0 of 1,000 at each of 70K, 80K, 90K and 100K (Wilson 95% upper bound 0.38%). At every smaller measured panel size all audits reject. These repetitions resample a fixed pool; they are not independent training runs.

In the contest: repeated attempts. The bounds above are for one audit. Let β\beta be the per-audit false-positive bound: aa for the permutation detector, 4e−τ4e^{-\tau} for the majority detector. With mm registrations and qq audits per registration, the probability that any false claim succeeds is at most min⁡{1,βmq}\min\{1,\beta mq\}.

Both counts are limited in practice. Every registration and audit request is an authenticated, permanent ledger record (Def. 3), so attempts are public and countable. In a contest only claims released before the target are admissible, which fixes mm once the target is released. A deployment can then cap the number of audits per claim and target, charge a fee per request, or fix a cap qq in advance and require significance at level a/qa/q for every audit, which keeps that claim's overall false-positive rate at most aa. It can also lower aa or raise τ\tau outright; for the majority detector the false-negative argument then needs nκ2≥max⁡{128,16τ}n\kappa^2\ge\max\{128,16\tau\}, so the panel size grows linearly in τ\tau.

Reviewer_2bVnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a method for proving model ownership including when the model is finetuned, by exploiting loss-order dependence signal between training examples. To overcome false retrospective claims that can be fabricated by an adversary, the approach suggest cryptographic commitments on a public ledger. It studies whether an adversary can remove this ownership signal through fine-tuning while preserving model utility. The approach explouts the idea of palimpsestic memorization seconded with public ledger comittments to keep chronology.

Critical strengths
  • I think the problem of publicly proving model ownership is important and the work is well motivated.

  • The idea is interesting, and the authors correctly identify that statistical fingerprinting alone is insufficient because an attacker can post-hoc fabricate a matching ordered dataset. Binding the fingerprint to pre-training commitments and public chronology, while is a well known technique from the distributed systems community, is still interesting in this context

Critical weaknesses
  • The robustness analysis relies on a frozen-gradient, first-order linearisation to make the optimisation tractable. However, this might not really capture full fine-tuning process or nonlinear utility-preserving transformations. For example an adaptive adversary might use nonlinear updates and choose attack data after learning the protocol. The authors already acknowledge larger deviations at the largest replay scale. The approach also inherits weaknesses of palimpsestic memorization like signal strength variance by model/corpus. I also wonder if the models picked for the experimetnal setup are sufficient with only 2 non Pythia models.

  • The framing branding it as provable is a little misleading, giving the impression that this is garanteed verifiability, but the actual security goals are stated informally; false-positive/false-negative probabilities are not theoretically bounded for the full construction. The empirical robustness is valuable but not a proof of any guarantees.

Questions for authors
  • Can you provide evidence that the test has sufficiently low false-positive rates against independently trained models on similar data, which is essential for contestability soundness?

  • The paper optimises a linearized frozen-gradient attack under a coefficient cap and evaluates utility via a limited set of proxies. How much does this underestimate a full fine-tuning, LoRA, quantization/pruning/distillation?

2 comments

We sincerely thank you for your supportive review and insightful feedback. We are glad that you find the problem of publicly proving model ownership "important" and our work "well motivated", and that you agree statistical fingerprinting alone is insufficient. We address your concerns and questions below.

Q1 (false positives on independently trained models)

We tested whether the loss of Pythia-deduped models is correlated with Pythia’s training order. These models are trained independently, on a deduplicated version of the Pile and in a different order.

ModelPermutation p-value
Pythia-1.4B0.00947
Pythia-1.4B-deduped0.14677
Pythia-6.9B0.00031
Pythia-6.9B-deduped0.20655

At the 1% significance level, the original Pythia models show a significant correlation (true positives), while the independently trained deduped models do not (true negatives).

We evaluated all four models on a shared audit panel of 10,000 sequences from the Pile, and computed the permutation p-value of the Pearson statistic (99,999 permutations) without a reference model. This comparison was also made by Kuditipudi et al. (2025).

Q2 (Robustness to fine-tuning, quantization and pruning)

We agree that our frozen-gradient attack is a first-order approximation and may not capture full fine-tuning and retraining, but such first-order approximations are standard in the analysis of large language models, where full fine-tuning cannot be analyzed exactly. For example, prior work has used similar first-order approximations to study language-model fine-tuning (Malladi et al., 2023 and Xia et al., 2024).

Our evaluation studies direct updates to the released model weights. Output-only distillation into a separately initialized student falls outside this setting. Following your suggestions, we have tested the effects of quantization and pruning on Pythia-1.4B and Pythia-6.9B. Neither removes the signal, and the p-value stays significant in all cases.

TransformationPythia-1.4B<br> p-valuePythia-6.9B<br> p-value
INT8<=0.00001<=0.00001
INT4 (group size 128)<=0.00001<=0.00001
10% pruning<=0.00001<=0.00001
30% pruning<=0.00001<=0.00001

The loss-order correlation stays significant after INT8 and INT4 quantization and 10% or 30% pruning.

Unlike the table in Question 1, this table uses reference models.

The audit panel is chosen after the attack, so removing the fingerprint requires erasing the loss gap between used and unused data across the whole population, which we expect to be costly. At the Pythia-6.9B checkpoint, examples already seen during training have lower corrected losses than examples not yet seen. Among the seen examples, those encountered later also have lower losses (Appendix A, Figure 4).

W1 (Signal strength across model families)

While it is true that the strength of palimpsestic memorization can vary across models, our fingerprint uses both palimpsestic memorization and the loss gap between used and unused data, so it does not depend on palimpsestic memorization alone. To test whether the signal holds across model families, we continued training five pretrained language models—Pythia-6.9B, OLMo-7B, Qwen2.5-1.5B, Qwen3-1.7B-Base, and Llama-3.2-1B—using data from the FineWeb 10BT subset. The following table shows the permutation p-value of the Pearson statistic on raw loss, without a reference model (99,999 permutations, so the smallest reportable value is 0.00001):

Modelp-value
Pythia-6.9B<=0.00001
OLMo-7B<=0.00001
Qwen2.5-1.5B<=0.00001
Qwen3-1.7B-Base<=0.00001
Llama-3.2-1B<=0.00001

Over a short training trajectory of 1,563 steps on 50,000 training examples, the signal is strong in all model families, though the strength varies.

This extends the evaluation beyond Pythia, CrystalCoder and OLMo to Qwen and Llama, five families in total.

W2 (formal guarantees).

Thank you for the careful point on framing, and we agree that provable guarantees are important. Following your suggestion, we have added formal security definitions and learning conditions in the revised paper (Appendix B) with the derivations in our reply to Reviewer pkFC. At a high level, we separate the cryptographic guarantees from the empirical ones. Soundness (false positives) is proven: because Alice commits to her corpus before fresh public randomness fixes the training order, and the verifier checks every audited example against that commitment, an adversary cannot fit a dataset and order to a model after the fact. For a target model trained independently of the claim, the probability of false acceptance is at most the significance level (1% per audit) . If the model still satisfies the two learning conditions in Appendix B.3, Assumptions 1 and 2, the majority-vote detector misses it with probability at most 0.95%. Whether a given modification, such as fine-tuning, keeps the model within these conditions is an empirical question, which our robustness experiments address.

Thank you again for your supportive review. We hope our responses address your remaining concerns. We are happy to discuss any further questions.

Reviewer_RftoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes contestable ownership proofs for open-weight models, enabling developers to challenge lineage declarations that omit their models. It combines cryptographic commitments and public release records with a memorization-based fingerprint derived from training order. Experiments on six models study targeted frozen-gradient attacks and report substantial utility loss at the first tested points where detection becomes non-significant.

Critical strengths

The framework addresses a relevant problem in open-weight model attribution by connecting statistical fingerprints with publicly verifiable records and procedures for challenging incomplete lineage declarations. In addition, the evaluation covers six models across three families and explicitly targets the detection signal, providing useful evidence on its behavior under the studied attacks.

Critical weaknesses
  1. The detector uses loss differences relative to a reference model, so reference choice can affect both detection power and measured attack costs. However, the paper does not clearly specify the references used across all configurations or explain their selection criteria. It also does not examine whether the results remain stable across suitable, independently trained references.

  2. The attack is optimized to weaken the detection signal, without explicitly preserving model utility. Therefore, the observed performance loss may partly result from the attack design rather than a necessary cost of evasion. The paper does not evaluate alternative unlearning methods or attacks that explicitly balance signal removal and utility preservation. Although the authors limit their conclusions to the tested directions, such comparisons would clarify whether detection can be evaded with substantially less performance loss.

Questions for authors
  1. Which reference model and checkpoint are used for each experimental configuration, and how are suitability and independence from Alice’s randomized training schedule established? Have you evaluated other suitable references, and, if so, how much do detection results and measured removal costs vary?

  2. Were any attacks that explicitly preserve utility, or alternative unlearning methods, evaluated beyond the reported frozen-gradient updates? If so, how do their detection-evasion and utility trade-offs compare? What evidence helps distinguish damage caused by the chosen update strategy from the cost of suppressing the fingerprint itself?

  3. Carlini et al. (2019) used randomly generated canaries to measure training-data memorization. If such canaries are kept private and combined with the same pre-training registration and public-verification mechanisms, what additional benefit motivates choosing the training-order fingerprint? Is this benefit supported by analysis or comparative results?

Reference: Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., and Song, D. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. USENIX Security, 2019, pp. 267–284. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini

4 comments

We thank the reviewer for the thoughtful feedback and for recognizing the relevance of our framework. We address key questions below.

W1 and Q1 (reference model choice and independence)

We used the following reference models for our experiments:

Alice’s modelReference model
Pythia-1.4BPythia-1.4b-deduped
Pythia-2.8BPythia-6.9b-deduped
Pythia-6.9BPythia-6.9b-deduped
Pythia-12BPythia-12b-deduped
CrystalCoderLLM360/Amber
OLMo-7Ballenai/OLMo-1B-hf

Reference table. Each row pairs one of Alice’s models with the reference used for it. We use the publicly released reference checkpoints listed above. The permutation test assumes that the reference is trained independently of the random order used to train Alice’s model. We’ve now stated the exact model used in the experiment more clearly in the revised paper. Pythia-2.8B uses Pythia-6.9B-deduped as the reference model because the public Pythia-2.8B-deduped checkpoint behaves anomalously as also pointed out by Kuditipudi et al. (2025).

We would like to clarify the role of the reference model. In the submission, we evaluate existing public models (Pythia, OLMo and CrystalCoder), since we cannot pre-train models at this scale ourselves. For these models, the fingerprint must be detected over a long pre-training trajectory, where the signal is weaker, so we use a reference model to remove per-example difficulty from the loss. In our protocol, however, the reference model is optional. When Alice uses the fingerprint over a relatively short training trajectory, the signal is strong enough to be detected without a reference, as shown below. When a reference is used, it is chosen by an honest verifier from a predefined, fixed pool. We also found that a poorly matched reference, trained on very different data, can weaken the signal, so we recommend using either no reference or a closely matched one.

To evaluate reference-free detection, we continued training five pretrained language models (Pythia-6.9B, OLMo-7B, Qwen2.5-1.5B, Qwen3-1.7B-Base, and Llama-3.2-1B) on data from the FineWeb 10BT subset. We randomly partitioned 100,000 documents into 50,000 used examples and 50,000 unused examples. Each model was trained for one epoch (1,563 AdamW steps), and WikiText perplexity stayed within 10% of the original. The following table shows the permutation p-value of the Pearson statistic on raw loss, without a reference model (99,999 permutations, so the smallest reportable value is 0.00001).

Modelp-value
Pythia-6.9B<=0.00001
OLMo-7B<=0.00001
Qwen2.5-1.5B<=0.00001
Qwen3-1.7B-Base<=0.00001
Llama-3.2-1B<=0.00001

The signals are detected in all five models without a reference model. Each row is one model after continued training, tested on raw loss with no reference. A smaller p-value means stronger detection.

Q2 and W2 (Gap between the LP attack and utility-preserving unlearning)

We agree that part of the utility damage may come from our attack design. Following your suggestion, we tested a utility-aware version of our attack, which adds a constraint limiting the KL divergence from the original model on the attack panel. This did not reduce the damage: the final checkpoint had even worse WikiText perplexity than the unconstrained attack, likely because KL divergence on the attack panel does not predict WikiText perplexity well.

More importantly, we believe most of the cost comes from removing the fingerprint itself, not from our specific update strategy. In our protocol, the audit panel is chosen after the attacker has modified the model. With enough samples, the verifier’s correlation statistic is close to that of the whole population, so the attacker must remove the correlation across the entire corpus, not just on a few examples. In each execution of our training protocol, only half of the corpus is used for training, which creates a loss gap between used and unused data across the population. Closing this gap means changing the model’s behavior on the entire corpus, for example by learning on the unused half and unlearning on the used half, which is inherently costly for any update method. We also evaluated how benign fine-tuning and post-training affect the signal. At the 1% significance level, the fingerprint remained detectable in three of the four Pythia models after 98% more training steps. For OLMo-7B and CrystalCoder, it was still detectable after 10% and 33% more steps, respectively, and first became non-significant after 21% and 44%. All percentages are relative to each model’s baseline step count. See Table 2 in the revised manuscript.

Q3 (Training-order fingerprint vs. Private Canaries)

Thank you for this question. We introduce a protocol that lets a developer publicly challenge the declared lineage of an open-weight model when it omits their model. The protocol binds a fingerprint derived from the selection and ordering of ordinary training examples to pre-training commitments and public release records. During a challenge, any third party can verify the evidence using a fresh sample from the committed corpus. We formalize the security goals of these challenges and prove guarantees under explicit assumptions.

Carlini et al. (2019) study a different setting with a different threat model. Their canaries serve as a diagnostic tool for a model developer seeking to measure unintended memorization in a model the trainer controls, for privacy auditing. Our setting requires publicly verifiable attribution even when the party holding the model weights actively modifies them to suppress the attribution signal. We will add this discussion to the related work section of the revised paper.

Private canaries could be incorporated into our framework, but their suitability depends on their robustness to deliberate removal and on how disclosure is handled across audits. If the verification procedure reveals the audited canaries, exposed canaries may be subject to targeted unlearning, and disclosure thus reduces the pool of canaries available for future audits. A larger canary pool could mitigate this concern, but its effects on model utility and resistance to removal would require evaluation. In contrast, our fingerprint is spread across the ordinary training corpus: each audit reveals only a small random panel, and removing the fingerprint requires changing the model at the population level, as discussed in Q2. To illustrate the disclosure issue, we followed Carlini et al. (2019) and planted a very small set of private canaries in Pythia models, with almost no effect on perplexity. Targeted unlearning then removes the canary signal, with a perplexity increase of about 1% or less.

ModelCanary p-value (before)Canary p-value (after unlearning)PPL change
Pythia-1.4B0.000020.739+1.06%
Pythia-6.9B0.000010.421+0.30%

Small sets of canaries are unlearned cheaply, at about 1% perplexity loss. Compare the “before” and “after unlearning” columns. A small p-value means the canary is detected, and a large one means it’s gone. The last column is the utility cost.

We hope our responses and the new experiments address your concerns. If so, we would be grateful if you would consider raising your score. We are happy to answer any further questions.

Thank you for the clarifications and additional experiments. It partly addresses my concerns about the reference model, utility-preserving attacks, and canaries, and better motivates the proposed fingerprint. However, my main concern is still, particularly, how much of the observed utility loss comes from the attack design, and how much is necessary to remove the fingerprint. The KL-constrained experiment leaves this unresolved. Also, the population-level argument explains why an attacker must suppress the signal, but it does not show that doing so must reduce utility. I therefore retain my current rating.

Thank you for the quick follow-up. We are glad the clarifications and new experiments were helpful and better motivate the fingerprint. We believe the remaining point comes from a misunderstanding about what the paper claims, and we are happy to clear it up.

The paper’s main contribution is the contestable ownership protocol design. The robustness study supports our memorization-based instantiation with empirical evidence that the signal is not cheap to remove in practice. Even an adversary given the exact audit panel and the first-order-optimal update direction loses utility before detection fails, whereas a small canary set is unlearned at about 1% perplexity cost on the same models. We do not claim that removing the signal necessarily costs utility.

By “costly for any update method” we meant the attacker’s overall cost, which may include data and compute as well as utility. Table 2 illustrates this: benign continued training erases the signal only after 21–44% additional pretraining-scale steps for OLMo-7B and CrystalCoder. Stronger attacks are an interesting direction for future work, as in any empirical robustness evaluation. Thanks to your question, we have revised the wording in Section 3 and Section 5 of the PDF accordingly.

We hope this clears up the misunderstanding, and we would appreciate it if you would consider raising your rating. We are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.