acceptodds
Withdrawn from ICLR 2027

Will LLMs Eventually Become Indistinguishable? Rethinking Authorship Attribution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM authorship attribution aims to identify the source model of a generated output by leveraging specific characteristics in its outputs. Existing approaches implicitly assume that different LLMs present sufficiently distinguishable generation patterns. However, this assumption may be challenged by knowledge distillation, which explicitly transfers knowledge between models while potentially altering the source-specific signals used for attribution. In this work, we systematically study how knowledge distillation affects black-box LLM authorship attribution. Across two benchmarks, nine source classes, and seven representative attribution methods, we find that knowledge distillation consistently impairs authorship attribution, causing distilled student outputs to become increasingly confused with their corresponding teachers. Moreover, we find that retraining attribution models on post-distillation outputs recovers only part of the lost attribution accuracy, indicating that the degradation cannot be fully explained by distribution shift. Instead, knowledge distillation appears to reduce the intrinsic distinguishability between student and teacher outputs. To theoretically explain this phenomenon, we develop an information-theoretic framework showing that knowledge distillation reduces the mutual information between generated text and its source, and further characterizing how much of this source information remains accessible to an attribution model. Together, our findings reveal an overlooked challenge for LLM provenance in an increasingly interconnected LLM ecosystem.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Authorship Attribution, Knowledge Distillation, Mutual Information

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fs4EOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies how knowledge distillation affects LLM authorship attribution. The authors show that, after a student model learns from a teacher, existing attribution classifiers become much more likely to incorrectly identify the student's outputs as coming from the teacher. Retraining these classifiers on outputs from the distilled student recovers most of the lost accuracy. The paper also presents an information-theoretic analysis to explain why distillation makes the outputs of the two models harder to distinguish.

Critical strengths

Authorship attribution is a meaningful problem, and the experiments cover multiple models, attribution methods, and distillation settings.

Critical weaknesses
  • The experimental results are unsurprising and the theoretical contribution is weak. The proposed "intrinsic source information" is essentially the Jensen–Shannon divergence between teacher and student output distributions. Since distillation minimizes their KL divergence, the local decrease in JS divergence is unsurprising. This also does not yield that the models are becoming indistinguishable.
  • Poor performance from classifiers trained before distillation mainly demonstrates distribution shift, not intrinsic indistinguishability. The authors' own retraining experiments recover much of the lost accuracy. The remaining gap may simply reflect classifier limitations rather than a fundamental difficulty in distinguishing the models.
  • More fundamentally, the output distributions of two models becoming more similar does not make them indistinguishable. As long as their output distributions are not identical, they remain statistically distinguishable in principle, given sufficiently many independent observations.
Questions for authors

None

Reviewer_A97VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how knowledge distillation affects LLM authorship attribution (predicting which model generated a given piece of text). The authors show that after distillation from a teacher model, attribution models increasingly confuse student-generated outputs with those of the teacher. This is evaluated across several model pairs, datasets, and attribution methods. The authors also provide an information-theoretic framework to explain the observed degradation.

Critical strengths
  • I think the problem is interesting and relevant. Given how prevalent knowledge distillation has become in LLM development. It makes sense to study its implications for model attribution is an important question.
  • Comprehensive empirical analysis: the authors evaluate several model pairs and attribution methods, and investigate factors such as distillation data size, model family, and attribution model retraining and their impact on atrtibution. The theoretical analysis is somehow useful in making things concrete and grounding the findings.
Critical weaknesses
  • The main findings are not particularly surprising. Distillation explicitly encourages the student’s output distribution to become closer to the teacher’s. So one would expected that attribution classifiers would increasingly confuse the two. So I am not particularly convinced that the central observation is particularly novel or very interesting.
  • The paper focuses primarily on off-policy distillation, where the student learns from teacher-generated outputs or distributions. However, other forms of distillation are increasingly common in modern LLM setups, particularly on-policy distillation (OPD), where the loss is computed on student-generated trajectories i.e., with reverse not forward KL. It is unclear whether the findings generalize to this setting and I suspect that OPD might have a less severe effect on attribution. Evaluating this would significantly strengthen the paper.
  • While the information-theoretic framework is interesting, I am not sure it adds much beyond formalizing an intuitive consequence of distribution matching. I actually find it to complicate the paper unnecessarily since it does not yield additional insights or predictions that were not already apparent from the empirical observations.
Questions for authors
  1. How do the findings change under on-policy distillation? Would attribution degradation be less severe than under the off-policy setting?
  2. Does the theoretical framework provide any predictions about attribution degradation that can be independently validated experimentally?
Reviewer_vxddOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper asks whether knowledge distillation undermines black-box LLM authorship attribution, the task of identifying which model generated a given text. Its central claim is that it does. A distilled student’s outputs become hard to tell apart from its teacher’s, and this is not just a classifier going out of date.

Setup. The authors use two QA benchmarks, ASQA (text) and ScienceQA (multimodal). Each has nine source classes: six open-weight models from three families, two API models (GPT-6 Astra and MiniMax-M3), and a human class. They test seven attribution methods. Students are distilled from a teacher, mostly with sequence-level KD and LoRA, on 10% of the training questions by default.

Their findings are (1) that KD transfers attribution from student to teacher. (2) A teacher from a different model family causes larger drops in the student attribution. (3) Retraining on the student outputs recovers the loss.

Critical strengths

The RQs (particularly 1 and 2) are important and timely to study as more distilled models are being deployed in practice.

Critical weaknesses

1 - There are no surprising findings. It is clear that distillation from a teacher will transfer some traits. It is unclear whether distillation transfers knowledge or the stylistic presentation that changes attribution.

2 - The eval held-out validation set is in-domain, and thus the results are more predictable. We are also not aware if there may be overlap between some topics in the training and validation.

3 - The technical details about which distillation methods are used in Fig. 3 are not clear. Since the authors used different algorithms, this detail is important.

4- The Gemma-2-9b and Gemma-2-27b are already student and teacher models [1]. In fig 1, it is more surprising that there is a major difference in the first plot. It is more evident that the eval settings are responsible for the major attribution shift.

[1] Team, Gemma, et al. "Gemma 2: Improving open language models at a practical size." arXiv preprint arXiv:2408.00118 (2024).

Questions for authors

Please see critical weaknesses.

Additionally:

1 - The theory assumes shared param near the teacher. What about other family models as teacher/student pairs?

2 - Why did the models shift in RQ3 to Gemma-3? Why are the teacher/student pairs not held constant?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.