Adaptive Transfer of Representational Signatures for LLM Reliability Detection

Large language models (LLMs) are reported to be susceptible to reliability issues including hallucinations and reasoning errors. Existing quality detectors can predict unreliable outputs based on internal representations with decent accuracies. However, training these detectors often require substantial domain-specific labeled data, which limits their applicability to new domains. We propose to address this challenge by transferring representational signatures across related domains. More specifically, we build a statistical framework that first extracts low-dimensional signatures for each domain, then uses cross-fitting to learn composition weights of domain-specific predictors. The weights adapt to each source domain’s predictive relevance to the target domain. Our theoretical analysis in the random matrix regime characterizes how prediction error and weights depend on target sample sizes and the statistical differences across domains. Experiments across multiple hallucination datasets and open-weight LLMs show that, with only 25 labeled questions in the new domain, adaptive transfer improves hallucination-detection AUROC by 7.8%. For reasoning-step prediction, it improves AUROC by 7.7%. The learned representational directions can also be used for activation steering to improve answer correctness with limited degradation in generation quality.