TRAIL: LoRA as Traceable Parametric Memory with Trace Tokens

Deploying large language models (LLMs) in high-stakes domains requires two long-competing capabilities: efficient domain-adaptive fine-tuning and traceability of each prediction back to its supporting training instance. Parametric LMs adapt efficiently but offer no per-instance index, while non-parametric kNN-LMs preserve instance-level traceability through retrieval but with substantial inference cost. Recent parametric imitations of kNN-LM remove the datastore but discard its traceability by construction. We propose TRAIL (Traceable Parametric Memory with LoRA and Trace Tokens), which recovers the instance-level traceability of kNN-LM within an efficient parametric module. On top of a frozen LoRA-adapted LM, TRAIL introduces a small set of learnable special tokens, called trace tokens, trained contrastively so that the hidden state of each prediction's trace token identifies its source training instance. On finance domain data, TRAIL attains 18–33% higher identification accuracy with 60–70% lower test perplexity compared to parametric memory baselines. This is achieved with 2× fewer trainable parameters, 60× smaller disk footprint, and 242× faster inference than kNN-LM. Counterfactual retraining shows that the identified documents are causally responsible, so TRAIL offers a foundation for accountable deployment of domain-adapted LLMs, supporting per-instance attribution, copyright compliance, and error diagnosis.