- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper asks whether context reliance in retrieval augmented generation is encoded as a single, transferable linear direction. The authors build 15 contrastive datasets: 13 synthetic A/B template sets, a ConFiQA derived counterfactual QA set, and a template free dialog construction. They also build a RAGTruth derived set in which an LLM rewrites each annotated hallucinated span into a minimal source supported replacement. For each dataset and each layer of Llama 3.2 1B Instruct and Llama 3.1 8B Instruct they compute a Context Source Vector (CSV) as the mean activation difference between the context reliant and parametric member of each pair (contrastive activation addition). Two controls are used: 100 random directions (lower) and a CSV fitted on RAGTruth itself (upper). In domain probes separate the synthetic classes almost perfectly (LDA up to 0.98 at middle layers of 8B), but best layer transfer AUROC is only 0.56 to 0.68 (mean 0.62) for 1B and 0.59 to 0.70 (mean 0.65) for 8B, against random means of 0.57 and 0.58 (95th percentile 0.634 and 0.645) and upper controls of 0.77 and 0.84. The authors report that vectors that are nearly orthogonal or negatively aligned reach similar best layer AUROC, often at different layers, and conclude that context reliance is not captured by one canonical direction but by several partly confounded features at different depths.
Critical strengths
The evaluation sets a better bar than most steering vector detector papers. Scoring each vector against random directions whose best layer is chosen in the same way, and against an in domain upper control, shows that many "best layer" results are explained by layer selection (the random 95th percentile at 1B, 0.634, is close to the best template, 0.681), and that ten of the 15 templates peak at layers 0 to 3 where the signal is probably lexical. The paper also shows directly that near perfect in domain probe accuracy does not predict transfer (probe F1 near 1.0 while transfer AUROC is 0.59 to 0.69). Recommending these two controls as a minimum bar is useful for the area.
Critical weaknesses
I think the transfer test and controls are too weak to carry the weight placed on them. The differences between vectors are small relative to the noise (best layer AUROCs of 0.59 to 0.70 against a random 95th percentile of 0.645 at 8B on a test set of 1,166 pairs), and no confidence intervals or bootstrap are given for any vector, yet the paper interprets orderings among templates and layers. The "RAGTruth pair" task also differs from the stated goal: the detector scores the final token of a human annotated hallucinated span against the final token of an LLM rewritten correction, so a score could track surprisal, fluency or authorship of the span rather than reliance on context, and no simple baseline (token log probability, entropy, or the paper's own probe) is reported on the same pairs to show how hard the task is. The upper control is a mean difference vector on RAGTruth (0.77 and 0.84), not a trained probe, and the closest prior method (O'Neill et al., a linear probe on an observer model) and white box detectors such as ReDeEP are not compared on the same data, so the reader cannot tell if 0.62 to 0.65 is poor for this family or for the task. No causal test (steering or ablation) is run, though the paper itself notes that comparable vectors can degrade output in both directions. Only two Llama models are tested.
Questions for authors
1/ Please recompute the cosine geometry at each vector's best AUROC layer, and with sign aligned, and report whether the "orthogonal but equally good" pairs survive.
2/What are bootstrap confidence intervals for the best layer and mid layer AUROCs of each vector on the 1,166 RAGTruth test pairs?
3/How was the layer for "Best" chosen for fairness, given the test labels are used to pick it, for the random controls as well as the CSVs?
2 comments
Thank you for the clear response, I will keep my score.
Thank you for the feedback and for recognizing the value of the controls and the negative transfer results.
We do not have the capacity to run further experiments for this rebuttal. We will acknowledge the missing uncertainty estimates, baselines, and causal tests, and keep the conclusions within what the existing experiments establish.