acceptodds
Under review as a conference paper at ICLR 2027

A Released LoRA Adapter Names the Document It Was Trained On

Abstract

LoRA adapters are low-rank factors smaller in size compared to the base models they are fine-tuned on. Over 130,000 LoRA adapters are publicly posted on Hugging Face alone. Anyone who downloads an adapter receives it without knowing the data it was fine-tuned on. Prior studies of these weights have produced a scalar or a verbal description of the training data, plus a generative attack reports ROUGE-2 below 9% on the source text, thereby concluding that extraction is extremely difficult. In this work, we show that a released adapter identifies the exact document it was fine-tuned on. Our method retrieves that document from roughly 570k candidates for all 256 held-out adapters trained for five steps and shows a 0.01% false positive rate across millions of shortlisted negatives. In that same setting, greedy decoding reproduces no document verbatim, so identification does not require the text to be recoverable. On a second base model, we refit the method end-to-end and it again identifies every document, while decoding reproduces at most one of 256. We trace the signal to the gauge-invariant product . A kernel over dW alone with no base model, tokenizer or forward pass, ranks the training document at C-AUC 0.897 among 16 candidates. We also measure the bounds of this method. We see that by eighty steps greedy decoding can recover the document and training one adapter on several documents dilutes which one is named. More broadly, our work shows that publishing an adapter can disclose the document behind it and privacy assessments must extend from a model's outputs to its weights.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LoRA adapters, hypernetworks, weight-space interpretability, training data extraction, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XfF4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether a publicly released LoRA adapter can reveal the exact document used during fine-tuning. The authors propose a two-stage document identification method. In the first stage, they train a kernel ridge regression model on 32,768 shadow adapters to predict document TF-IDF representations from LoRA weight updates, which are then used to shortlist candidate documents. In the second stage, they rerank the shortlisted documents using the loss difference between the adapted model and the original base model.

Critical strengths
  • The paper investigates an interesting privacy problem: whether released LoRA adapters can reveal information about their training documents.
  • The paper demonstrates that training documents can be identified from released LoRA adapters using a two-stage method trained with 32,768 shadow adapters.
Critical weaknesses

Limited motivation and practical relevance of the threat model. The paper is motivated using huggingface LoRA adaptors and claims that "most systems train one adapter per document," (line 38) but the cited works (Su et al., 2025; Charakorn et al., 2026) focus on document-specific, RAG-like approaches rather than LoRA fine-tuning. Thus, the assumption that each adapter is trained on a single short document lacks sufficient practical justification.

Limited generalizability The main experiments consider a setting where each LoRA adapter is fine-tuned on a single document. The paper's own multi-document experiments, Table 3, show that identification of a specified training document degrades as the number of training documents increases (the success rate drop from 100% (one document) to 10%(8 document)). This raises concerns about how broadly the main findings generalize to more realistic fine-tuning settings.

Unclear comparison with existing baselines* First, in table 1, the comparison between reconstruction based method is not entirely fair, as these methods reconstruct training text without access to the target document. Second, Table 2 compares the proposed method with existing MIA under substantially different experimental settings. In particular, their full two-stage methods train the adaptor on a single-document and use 32,768 shadow adapters, while LiRA/RMIA are evaluated on adapters trained with 32 documents and 255 shadow adaptors...

Unclear novelty. The paper highlights identification without model execution as a key contribution. However, the main 100% identification results rely on a two-stage pipeline that requires running both the base and adapted models for loss-based reranking. The paper should clarify the novelty of the proposed approach beyond existing loss-based attacks and distinguish the capabilities of the weights-only method from those of the full pipeline.

Questions for authors
  1. Could the authors evaluate the proposed two-stage method against LiRA/RMIA under the same fine-tuning setup, candidate pool, shadow model budget, and evaluation metrics? In particular, how do these methods compare when the experiment setup are aligned.

  2. Since the full identification pipeline relies on loss-based reranking, what is the advantage of the proposed weight-based retrieval stage over directly applying existing loss-based membership inference methods? especially when accounting for the cost of training shadow adapters.

  3. Could the authors provide further evidence that one-document-per-adapter fine-tuning is common among publicly released LoRA adapters?

Reviewer_2HKhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a released LoRA adapter (factor pair A, B) can be used to identify the specific document it was fine-tuned on. The setting is one adapter per 150-word document (wikitext-103 windows from distinct articles), Gemma-2-2B-it with rank-8 LoRA on mlp.down_proj, lr 2e-3, batch size 1, trained for either 5 steps (held-out perplexity 1.04× base; greedy decoding reproduces nothing) or 80 steps (100× base; greedy decoding reproduces every document verbatim). Two auditors are considered. (i) An auditor with the base model, 32,768 shadow adapters trained with the victim's recipe, and a candidate pool of 579,896 documents runs a two-stage attack: stage one fits kernel ridge regression from the GL(r)-invariant Frobenius kernel over ΔW = AᵀB to a 20k-dim tf-idf vector and shortlists 10,000 candidates; stage two rescores the shortlist by the loss difference L_adapted(d) − L_base(d). It ranks the true document first for all 256 held-out victims at both 5 and 80 steps, on Gemma and (refit) on Qwen2.5-3B-Instruct, with a single threshold giving TPR 1.0 at 0.01% FPR over 2.56M shortlisted negatives, and it survives adding 34,544 near-duplicate windows from the victims' own articles. (ii) A weights-only auditor with only the adapter file and 224 shadow adapters uses the same kernel ridge to rank the document against 15 distractors at AUC 0.897 (chance 0.5; 0.40 above a shuffled-pairing null), without any forward pass at method time. Further analyses show the signal lives in the largest principal directions of the update population, that identification degrades to ≈1/k when an adapter is trained on k documents (top-1 0.105 at k=8; chance-level weights-only AUC at k=32), that post-hoc noise/rank truncation/quantization down to 3 bits do not remove the signal, that DP-SGD noise reduces direct kernel matching but not the refit probe without large utility loss, and that an owner can detect exposure with four forward passes. Several pre-registered predictions failed and are reported as such. Code is linked anonymously.

Critical strengths
  1. A rigorous, honest, and reproducible empirical case that released adapters leak document identity without extractability. The 256/256 top-1 result over 579,896 candidates at 5 training steps (Table 1, row C vs. D1), replicated end-to-end on a second base model (App. F.3) and robust to 3,040 near-duplicate negatives overlapping the victim window by a median of 110/150 words (App. C.1), is a clear finding with a clear message for privacy assessment of open-weight adapters. This is consequential because the closest prior work (DistilledPRAG, ROUGE-2 < 9%) concluded extraction was intractable; this paper shows that the wrong quantity was being measured. The methodology is unusually transparent: pre-registered predictions that failed (position kernel on the contiguous pool, k=32, fixed-FPR thresholds, base-loss correlation) are all reported, which raises my trust in the numbers that did hold.
  2. The weights-only probe and its analysis are a genuine, if modest, addition. Ranking documents from ΔW alone with no base model at method time (AUC 0.897 among 16; 0.880 against all 1,572 pool documents), together with the principal-direction ablation (removing the largest 111 directions pushes AUC below chance; random directions of equal mass do not) and the perturbation study (3-bit quantization and rank-2 truncation leave Rs ≈ 1), gives the community a measurable handle on what in the weights carries the document.
Critical weaknesses
  1. The claim's scope exceeds the evidence: identification works only for single-document adapters trained with a known recipe, yet the title, abstract and motivation (130k Hugging Face adapters, copyright audits) are stated unconditionally. All headline results are k=1. By the paper's own Table 3 / App. E.1, top-1 accuracy on the target falls to 0.469 at k=2 and 0.105 at k=8 (≈1/k; the interval does not clear the pre-registered 0.10 floor), at k=32 the target sits at median rank 153,724 after stage one, and the weights-only probe is at chance from k=8. The attacker also needs the victim's rank, target modules and learning rate to train matched shadow adapters, and a 15× learning-rate mismatch costs 0.20 AUC (Table 9). Virtually no publicly posted adapter is trained on one 150-word document; the one-document regime comes from parametric RAG / Doc-to-LoRA, which is a legitimate but much narrower deployment. This matters for the recommendation because a reader of the title and abstract would come away believing something the experiments do not show; the claims need to be restricted to the single-document (or few-document, "names one of the k") regime, and the Hugging Face framing softened accordingly. This is fixable by rewriting, which is why I do not rate lower.
  2. The identifying mechanism is standard calibrated-loss membership inference, and the necessity of the paper's main algorithmic novelty (the weight-space sieve) is not established. Stage two is s(d) = L_adapted(d) − L_base(d), the loss-calibration score of Yeom et al./Carlini et al., and App. C.2 shows an RMIA-style score is statistically indistinguishable from it. App. D.1 states that for every shortlisting method, top-1 accuracy equals shortlist recall: whenever the document is in the shortlist, stage two finds it. So the contribution reduces to stage one's recall advantage over cheaper shortlists (206/256 for the best decoding baseline on Gemma; 252/256 on Qwen when handed the first token). The case for stage one rests on the claim that scoring all 579,896 candidates needs 148M forward passes (825 GPU-hours) at batch size 1 over full 150-word documents. A sieve only needs recall, not exact losses, so batched (e.g., 64) prefix-only scoring (e.g., first 32 tokens) under the adapted and base models would plausibly raise throughput by 20–50× and scan the full pool in tens of GPU-hours total, with no 32,768 shadow adapters, no 853 minutes of offline kernel construction, and no dependence on knowing the victim's recipe. The paper's reason for rejecting batching (loss changes at 27/128 positions, 17/8,128 orderings flipped) applies to exact rescoring, not to a recall-oriented sieve. Without this comparison I cannot tell whether the GL(r)-invariant kernel is needed or merely elegant. In absolute terms the weights-only probe is also weaker than its AUC suggests: at 0.1% FPR it detects 7/256 and at 1% FPR 43/256, missing all three pre-registered thresholds (App. B.2).
Questions for authors

Clarification requests that could change my assessment:

  1. Please run a batched, prefix-only brute-force sieve (e.g., batch 64, first 16–32 tokens, score L_adapted − L_base over all 579,896 candidates) and report its recall@10,000, wall-clock, and the resulting end-to-end top-1 after stage two. If this matches stage one's recall at comparable or lower cost, what is the remaining role of the kernel sieve?
  2. Please state explicitly, in the abstract and introduction, that the identification results are for adapters trained on a single document (or report what the two-stage method achieves on realistic multi-document adapters, e.g., k ∈ {64, 256} or an adapter trained on a few thousand documents): does it still rank any training document first, and at what FPR?
  3. How is the two-stage attack affected by recipe mismatch (Table 9 covers only the weights-only probe)? In particular, if the auditor does not know the learning rate or the adapted modules, what happens to stage one's recall?
  4. For the 0.01% FPR threshold: how many false positives per victim does this correspond to on the 10,000-candidate shortlist, and on the full 579,896 pool? What does an auditor do with them?
  5. Are the 32,768 stage-one shadow documents inside the 579,896 candidate pool? If so, does the ridge ever rank a shadow document (which it has seen as a training target) highly for a victim?

Suggestions (not affecting the score): report the k>1 results on Qwen as well; add privacy accounting (ε) for the DP-SGD experiment or label it clearly as a noise sweep; consider moving Table 4 (what each attacker holds) into the main text.

Reviewer_2QMMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses an important question whether a released LoRA adapter discloses the document it was fine-tuned on. The proposed two-stage method retrieves the exact training document from a large corpus of candidates with minimal false positive rate. The method is refit end-to-end on a second base model (Qwen2.5-3B-Instruct) and again identifies every document. The paper also measures the bounds of the method and conclude that privacy assessments must extend from a model's outputs to its weights.

Critical strengths
  1. This paper challenges prior work that recovering document content from LoRA weights is extremely difficult. Using a public base model and a released adapter, the proposed two-stage method ranks the exact training document first among 579,896 candidates for every held-out adapter on Gemma-2-2B-it and Qwen2.5-3B-Instruct.

  2. The paper shows that evaluating parameter privacy through generative inversion alone provides false reassurance.

Critical weaknesses
  1. The claim of using 130000+ hugging face adapters is not followed through in the evaluation.
  2. The weights-only probe is evaluated among 16 candidates and at 0.1% and 1% false-positive rates it detects 7 and 43 of 256 targets which are both short of the pre-registered rates, and it scores AUC 0.499 on 32-document adapters. Can you qualify the "Identification without running a model" contribution and give the stage-one-alone top-1 over the full pool.
  3. DP-SGD experiments use lot size one without privacy accounting and the contiguous-window pool, which section 5.3 shows let position pass for content, but the section 5.2 account is stated as untested. It would be good to rerun mitigation on the distinct-article pool with reported privacy budgets.
  4. How often does the fixed stage-two threshold fire when the training document is absent from the candidate pool, or present only as a near-duplicate or paraphrase? The near-duplicate test is reported at 80 steps. For example, does it also hold at 5 steps?
  5. The paper is hard to read.
Questions for authors

See the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.