- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether a publicly released LoRA adapter can reveal the exact document used during fine-tuning. The authors propose a two-stage document identification method. In the first stage, they train a kernel ridge regression model on 32,768 shadow adapters to predict document TF-IDF representations from LoRA weight updates, which are then used to shortlist candidate documents. In the second stage, they rerank the shortlisted documents using the loss difference between the adapted model and the original base model.
Critical strengths
- The paper investigates an interesting privacy problem: whether released LoRA adapters can reveal information about their training documents.
- The paper demonstrates that training documents can be identified from released LoRA adapters using a two-stage method trained with 32,768 shadow adapters.
Critical weaknesses
Limited motivation and practical relevance of the threat model. The paper is motivated using huggingface LoRA adaptors and claims that "most systems train one adapter per document," (line 38) but the cited works (Su et al., 2025; Charakorn et al., 2026) focus on document-specific, RAG-like approaches rather than LoRA fine-tuning. Thus, the assumption that each adapter is trained on a single short document lacks sufficient practical justification.
Limited generalizability The main experiments consider a setting where each LoRA adapter is fine-tuned on a single document. The paper's own multi-document experiments, Table 3, show that identification of a specified training document degrades as the number of training documents increases (the success rate drop from 100% (one document) to 10%(8 document)). This raises concerns about how broadly the main findings generalize to more realistic fine-tuning settings.
Unclear comparison with existing baselines* First, in table 1, the comparison between reconstruction based method is not entirely fair, as these methods reconstruct training text without access to the target document. Second, Table 2 compares the proposed method with existing MIA under substantially different experimental settings. In particular, their full two-stage methods train the adaptor on a single-document and use 32,768 shadow adapters, while LiRA/RMIA are evaluated on adapters trained with 32 documents and 255 shadow adaptors...
Unclear novelty. The paper highlights identification without model execution as a key contribution. However, the main 100% identification results rely on a two-stage pipeline that requires running both the base and adapted models for loss-based reranking. The paper should clarify the novelty of the proposed approach beyond existing loss-based attacks and distinguish the capabilities of the weights-only method from those of the full pipeline.
Questions for authors
-
Could the authors evaluate the proposed two-stage method against LiRA/RMIA under the same fine-tuning setup, candidate pool, shadow model budget, and evaluation metrics? In particular, how do these methods compare when the experiment setup are aligned.
-
Since the full identification pipeline relies on loss-based reranking, what is the advantage of the proposed weight-based retrieval stage over directly applying existing loss-based membership inference methods? especially when accounting for the cost of training shadow adapters.
-
Could the authors provide further evidence that one-document-per-adapter fine-tuning is common among publicly released LoRA adapters?