Do Context-Reliance Directions Transfer to Hallucination Detection in RAG?

Representation engineering identifies directions in transformer activation space that can be used to measure or influence model behavior at inference time. In Retrieval-Augmented Generation (RAG), such a direction would be an attractive hallucination signal: it is intrinsic, costs one dot product per token, and uses information that post-hoc detectors discard by treating the generator as a black box. We ask whether context reliance is encoded as a single, transferable linear direction. We construct 15 contrastive datasets spanning hallucination-prone scenarios — 13 synthetic A/B templates, an external counterfactual QA corpus, and a template-free dialog construction — and extract one Context-Source Vector (CSV) per dataset from Llama-3.2-1B-Instruct and Llama-3.1-8B-Instruct. Each CSV is then scored on RAGTruth against two controls: 100 random directions (lower control) and a CSV fitted in-domain on RAGTruth (upper control). The answer is negative in an informative way. CSVs do transfer above chance, but only modestly: best-layer AUROC is 0.56–0.68 (mean 0.62) for the 1B model and 0.59–0.70 (mean 0.65) for the 8B model, against random-direction means of 0.57 and 0.58 and in-domain upper controls of 0.77 and 0.84. Crucially, CSVs that are nearly orthogonal or even negatively aligned with one another achieve indistinguishable detection performance, yet often peak at entirely different layers. Effective context-reliance signals are therefore not uniquely represented by a single activation-space direction; instead, the extracted directions appear to capture distinct, partly confounded features that reside at different depths in the network and are similarly predictive downstream. The larger model encodes more of these features, yielding a small but consistent improvement in transfer.