acceptodds
Under review as a conference paper at ICLR 2027

Controls Before Claims: Auditing Truth and Context-Faithfulness Steering Directions

Abstract

Large language models can produce fluent answers that rely on false information or ignore the evidence in the prompt. We study truthfulness and context faithfulness through directions in activation space. We extract a truthfulness direction, a context-presence direction, and a faithful-answer direction, and we evaluate them on five models (Llama 3.1 8B Instruct and base, Qwen 3.5 27B, Qwen 2.5 7B, and Mistral 7B) against controls that the steering literature usually omits: norm-matched random directions, dose matching, layer sweeps, extraction-source ablations, and judge-scored informativeness. On every model, a context direction steers context-following under knowledge conflict far above random directions of the same size (up to +72 points on Llama 3.1 8B Instruct), with informative answers, and the effect is invariant to the extraction data. Which contrast works is model-specific: the faithful-answer direction dominates on Llama and Qwen 2.5, the presence direction on Mistral. The truth direction reads whether the supplied context is false before generation on every model, but steers truthfulness only when it is polarity-balanced and applied at the layers where a negation control shows it encodes truth (TruthfulQA MC1 0.51 to 0.59, above 30 random directions); at the layer the standard calibration selects it only suppresses answers. After whitening, the truth and context directions share no geometry. These results show that direction-based claims depend on controls and layers that are rarely reported; they do not show that any single direction is a reliability intervention.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, representation engineering, evaluation methodology, random baselines, truthfulness, context faithfulness, knowledge conflict, retrieval-augmented generation, large language models, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dGfUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper takes three common steering directions (one for truth, two for following the given passage), applies them to five open models, and asks two questions: does the effect beat a random vector of the same size, and what else changes? Context steering does beat random, but which version works best differs by model, and it makes models answer questions the passage can’t answer, and sometimes lowers truthfulness. The usual truth direction does not beat random at the settings validation picks, and mostly just makes the model decline to answer. A better-built truth direction, applied at later layers, helps only on Llama, and even there it makes the model get more answers wrong when the passage is correct. Takeaway: a gain on the metric you tuned for is weak evidence on its own.

Critical strengths
  • The paper makes a convincing case that steering work should routinely report matched random-vector baselines and ablations (layer sweeps, dose comparisons, extraction-source checks) alongside the behaviours the selection metric doesn’t capture, rather than only the gain on the target metric. Given how many papers have reported truthfulness and context directions on the strength of a single tuned metric, this is a welcome step back and the kind of plausibility checking the area has needed. The controls are also important, they change conclusions: the calibrated truth direction’s MC1 gain sits inside the random-vector band, and the negation test explains why the validation-chosen layers fail.

  • The authors check their own work carefully. They flag 14% of ConFiQA test items as possibly defective and re-score without them, have two annotators blind-check the automatic judge and report agreement, admit they could not reproduce ContextFocus’s published gain, and state plainly when a setting was chosen on test data. This is the same standard they ask of others, applied to themselves.

Critical weaknesses
  • The evidence is strong for one model and thin for the others. Llama 3.1 8B Instruct gets 50 random vectors; the remaining models get 5–10. The “five models” framing is really one well-tested model plus four lightly tested ones. The random vectors are also the simplest kind (isotropic), which the authors concede is a weak null in anisotropic activation space.

  • The truthfulness metric is shaky. The paper uses MC2 candidate sets for MC1, so chance is already ~0.45 and the numbers are not comparable to published scores, and “I have no comment” is a true candidate on 80 of 616 questions, so the metric can be gamed by declining (acknowledged in the paper). This weakens both the negative result (the standard truth direction doesn’t help) and the positive one (the later-layer direction helps on Llama). The natural remedy, judging free-form answers, has only moderate human agreement on informativeness and was not applied to the later-layer t_G answers at all, so the one positive truth result is the least independently checked.

Questions for authors
  • Would the main conclusions survive stronger nulls? In particular, if random directions were drawn with the covariance of the activations (or restricted to the top principal subspace) rather than isotropically, would the context directions still exceed every matched random vector on the four lightly tested models?
1 comment

Thank you for the review, and for the question on stronger nulls, which led to the most informative new experiment.

1. Would the conclusions survive stronger nulls? In rank, yes; in margin, only partly. For each model we drew 20 random directions from the covariance of the unsteered hidden states at the steering layer and 20 from its top 50 principal components, scaled them to the length of the stronger context direction, and evaluated them on the full test set. The learned direction exceeds every one of the 40 on all five models, so the literal answer to your question is yes. But on four models the margin is small: the best structured vector comes within 4 points of the learned direction on Llama Instruct, Llama base, Qwen 2.5 and Mistral, where the best isotropic vector stays at least 10 points behind. A paired comparison on the same items still separates the learned direction from the best structured vector on Llama Instruct, but not on the other three. Only on Qwen 3.5 do structured vectors behave like isotropic ones. On Mistral the structured vectors that help most are partly aligned with the learned direction; on Qwen 2.5 and Llama Instruct the best top-50 vectors are almost orthogonal to it. We report this as a result in Section 4, and the abstract no longer says without qualification that context steering exceeds all matched random vectors.

2. Evidence was thin outside one model. Every model now has 25 isotropic random vectors for each calibrated direction, scored on the same test items as the learned directions. Llama Instruct keeps the two 50-vector families of the submission in addition. The stronger context direction exceeds all of them on every model. The calibrated truth direction does not exceed them on MC1 on any model.

3. The truthfulness metric. We ran three checks (Appendix I, Tables 14 and 15).

  • Standard single-answer MC1: the later-layer direction keeps its gain on Llama Instruct and a little over half of it on Llama base. The other models show none. The calibrated direction shows no significant change on any model, so the negative result stands.
  • Split by whether "I have no comment" is a true candidate: the Llama Instruct gain holds on the 536 questions that do not reward declining. About two thirds of the Llama base gain comes from the other 80.
  • Layer and dose chosen on validation questions instead of test questions: the Llama Instruct gain shrinks by about a fifth, and the conclusion for each of the five models is unchanged. On the two Llama models the direction exceeds all ten random vectors matched to these settings, by a smaller margin on Llama Instruct.

The positive truth result is therefore now stated for Llama Instruct, with Llama base as a weaker, qualified case.

One correction on the free-form check: the judge was applied to the later-layer answers (Section 7 reports that it labels fewer of them informative than unsteered answers; Table 11 gives the values, together with a new coherence label, under which more of them are coherent than unsteered answers). What is missing is a human audit of those labels, and we agree that this leaves the positive result less independently checked than the others. The limitations say so.

Reviewer_1HE5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper evaluates steering methods for truthfulness and context following. It finds that their effectiveness varies across models and settings, and that better target scores can come with undesirable changes in other behaviors. For example, steering that improves context following can also make models more likely to answer when the supplied passage lacks the necessary information.

Critical strengths
  • S1. Timely and interesting evaluation question. The paper examines whether improvements in steering metrics reflect intended behaviors, providing useful baselines and evaluation methods for assessing these improvements.
  • S2. Extensive and clearly documented experiments. The paper presents a broad set of experiments, with individual results clearly presented and experimental conditions documented to support replication.
Critical weaknesses
  • W1. Potential confounding from extraction position and answer conditioning. The vector constructions differ not only in their intended contrast but also in where hidden states are extracted and what information is available. The context-presence vector is extracted before an answer, whereas the faithful vector is extracted after a supplied answer. These differences introduce a potential confound, making it difficult to separate the effects of extraction position and answer conditioning from those of the intended semantic contrast.

  • W2. Incomplete random-control evaluation.

    • The number of random vectors varies across conditions in Table 6, with some conditions using only ten. The authors should use a consistent, sufficiently large number of random vectors to characterize random intervention effects and support reliable comparisons.

    • Each learned steering vector should be compared with random controls matched to its own intervention layer and magnitude (Table 6, faithful and context cases). Controls from a different family do not provide a matched baseline when these settings differ, because differences in intervention strength or location could affect the results. These matched comparisons should be reported for every evaluated steering family and model.

  • W3. Incomplete validation of generated-answer quality. Table 1’s answer-string matching does not establish that generated responses remain coherent and meaningful. Its difference score can also improve from a negative baseline when the model stops meaningfully producing either answer. The perplexity constraint provides a very indirect safeguard. It is measured on another dataset, and the metric itself, even combined with baselines, does not guarantee coherent, meaningful generations.

  • W4. Fragmented presentation and limited depth of insight. The paper reads like a sequence of experiments reported in tables, each followed by a description of the observed results. Connections between these evaluations are limited, and the authors do not sufficiently develop a connected argument across sections, leaving the paper without a strong central narrative. Although the overarching evaluation theme is clear, the analysis remains largely sparse. It documents a range of effects but offers limited explanation of why they arise, with little development of these observations into broader insights about steering.

Questions for authors

Please see the Weaknesses section.

2 comments

Thank you for the detailed review. We ran new experiments for W1 to W3 and rebuilt the framing for W4.

W1. Extraction position and answer conditioning. We agree: the submission could not separate the contrast from the position at which it is read. We built the two missing directions from the same examples, the presence contrast read after an appended answer and the answer contrast read before generation, and steered with all four at shared layers and equal doses on four models (Appendix J; Qwen 3.5 is not in this revision). The control changed one of our claims. On Llama Instruct, the large gap between the two directions in Figure 2 comes mostly from the layer. At the layer where calibration placed v_faithful, the presence contrast recovers most of its effect; at the layer where calibration placed v_context, none of the four directions does much. Reading position matters on each of the four models, but not in one direction. On Mistral, at the calibrated layer, both contrasts work better when read before generation. On Llama base the opposite holds: read at the appended answer, the presence contrast is about as effective as v_faithful, while the answer contrast read before generation does not help at the calibrated layers. Figure 2 is now described as a comparison of calibrated settings, not of contrasts. One limit is stated in the appendix: the answer contrast read before generation still shows a proposed answer, so the control does not separate the contrast from the presence of an answer in the prompt.

W2. Random controls. Every calibrated direction on every model now has 25 random vectors at its own layer and length, scored on the same test items as the learned direction (revised Table 1 and Table 7). Llama Instruct keeps its two earlier 50-vector families as well. The stronger context direction exceeds all 25 on every model. In Table 2 the abstention controls now use ten random vectors at the layer and length of the listed direction, including v_context on Mistral (three for Qwen 3.5, whose run is in progress); Table 22 adds both context directions of every model. The equal-dose and layer comparisons still use five or fewer random vectors per setting, and the limitations say so. Following Reviewer dGfU we also drew random directions from the hidden-state covariance. These are a much harder control: on four models the best of them comes within 4 points of the learned direction, and on three of these a paired test on the same items cannot separate the two. The abstract now reports how close they come.

W3. Quality of the generated answers. Your concern was that the score can rise when the model stops giving either answer. We now report the four parts of the score under the stronger direction of every model (Table 19). The share of answers that give the passage's answer rises on every model, further than under any of the 25 matched isotropic random vectors, and the share that give neither answer falls on four models and rises by one point on the fifth. We also asked the judge a third question, whether each answer is fluent text that responds to the question. Under the stronger direction the share of coherent answers rises on all five models, and on three of the four models with random vectors of the same length it rises more than under any of them (Table 20). We agree that string matching and the perplexity limit are indirect; the paper now rests answer quality on the parts of the score, the judge's informativeness and coherence labels, and the human audit of the judge.

W4. Presentation and depth. The introduction and the discussion now state one argument: each stricter control removes part of what a weaker one would credit to the learned direction. Against random vectors of the same length only the stronger context direction of each model survives; structured random directions reproduce most of that effect on four of five models; and with layer and reading position held fixed, most of the gap between the two context constructions on Llama Instruct disappears. We also rewrote the results paragraphs of Sections 4 to 7 to state each finding in words; the rewriting changed no result, and sentences cite the table or figure that holds their values. The new controls also suggest an explanation the submission lacked: much of what these directions do is to push the hidden state, at the right layer, along directions the model already uses, and the particular contrast adds less than the calibrated comparison suggests.

I thank the authors for their response and for the updated version of their work. I think my concerns are sufficiently addressed by the response, and I updated my evaluation to recommend a weak acceptance of the work.

Reviewer_mhbdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether the gains of standard mean-difference steering directions for truthfulness and context faithfulness survive proper controls. Across five models from three families, it compares a truth direction and two context directions with random vectors of the same length at the same layer, and measures MC1, FaithEval abstention and answer informativeness alongside the selection metric. The context directions hold up well: on every model the stronger one beats all tested random vectors, but which construction is stronger differs across models, and the gains can cost abstention or MC1. The truth direction fares worse. Its projection separates original from counterfactual passages better than random projections, yet at its validation-selected setting it does not raise MC1 beyond random vectors, and on Llama 3.1 8B Instruct it mainly suppresses answers. A polarity-balanced truth direction at later layers does raise MC1, but only on the two Llama models, and on Llama Instruct the gain comes with lower accuracy when the passage is correct.

Critical strengths

S1\textbf{S1} The authors makes a useful methodological point: a gain on the selection metric is weak evidence that steering actually improved the model. By combining matched random baselines, equal-dose comparisons and off-target evaluations across several model families, it shows concretely how these controls change the conclusions, and offers a practical template for evaluating steering methods.

Critical weaknesses

W1\textbf{W1}The paper provides a useful audit, but its methodological contribution is largely incremental: random-vector controls and off-target evaluations already appear in prior and concurrent work (Li et al., 2023; Ferrando et al., 2025; Malla et al., 2026). The main advance is applying these controls jointly to truth and context steering across model families. Its empirical scope is also limited to mean-difference directions, leaving applicability to other steering methods such as SAE-based steering untested.

W2\textbf{W2} The random-vector controls support the large context-score effects, but they are weaker for statistical claims on the non-Llama models and for attributing off-target effects to a specific direction. Context-score random effects are generally evaluated on seed-specific 750-item slices rather than on the same evaluation set as the learned directions, and several calibrated and off-target comparisons on the non-Llama models use only 3 to10 isotropic vectors, making these comparisons primarily descriptive. Most importantly, Mistral's calibrated vcontextv_{\mathrm{context}} has a relative dose of 0.63, whereas the abstention controls in Table2 use the smaller norm of vfaithfulv_{\mathrm{faithful}} (dose 0.34). Figure2 provides a dose-matched control only for the false-context score, not for abstention. The observed abstention drop is therefore real, but it is not yet shown to be specific to vcontextv_{\mathrm{context}}.

References\textbf{References}

Ferrando, J., Obeso, O., Rajamanoharan, S., and Nanda, N. (2025). Do I know this entity? Knowledge awareness and hallucinations in language models.

Li, K., Patel, O., Vi{'e}gas, F., Pfister, H., and Wattenberg, M. (2023). Inference-time intervention: Eliciting truthful answers from a language model.

Malla, S., Choi, C., and Choi, J. H. (2026). Steering interference reflects the model's defaults, not the behavior directions.

Questions for authors

Q1\textbf{Q1} Since "I have no comment." is a true candidate and never a false one on 80 of the 616 TruthfulQA questions, could the authors report the MC1 change separately on these 80 questions and on the remaining 536?

2 comments

Thank you for the careful reading. We take your points in order of weight.

1. The Mistral abstention control is now matched to v_context. You are right that Table 2 compared v_context with random vectors that were shorter than it. We ran ten random vectors at the layer and length of v_context itself, on the same FaithEval items. None of them lowers abstention below 15% (unsteered: 30%); v_context lowers it to 4%. The drop is specific to the direction. Table 2 now reports the matched range and the footnote is gone. A second observation points the same way: on Llama Instruct and on Mistral the weaker of the two context directions does not lower abstention; it raises it against the unsteered model (Table 22).

2. Random vectors are now scored on the same items, 25 per direction, on every model. Each calibrated direction on each model has 25 isotropic random vectors at its own layer and length, evaluated on the same test items as the learned direction (revised Table 1; Table 7 in the appendix). On Llama Instruct they come in addition to the two 50-vector families of the submission, which stay in the appendix. The stronger context direction exceeds all 25 on every model, and the lower end of its bootstrap interval lies above the best random vector in each case. The calibrated truth direction does not exceed the best of its 25 random vectors on MC1 on any model. Two statements in the submission did not survive and are corrected: the margin on Qwen 3.5 is 7.5 points, not "more than 10", and on Mistral the weaker direction v_faithful now falls inside its random band.

3. MC1 on the questions that reward declining, and on the others. For the later-layer direction t_G at the settings of Table 4. The two groups have 80 and 536 questions; for Qwen 3.5, which uses 400 questions there, 49 and 351.

Modelall questions"no comment" is a true candidateother questions
Llama Instruct+0.080+0.175+0.065
Llama base+0.068+0.338+0.028 (interval includes zero)
Mistral+0.010+0.050+0.004
Qwen 2.5−0.010+0.025−0.015
Qwen 3.5+0.018−0.122+0.037

On Llama Instruct the gain holds on the questions that do not reward declining, and random vectors at the same layer and dose gain essentially nothing there. On Llama base it does not hold: about two thirds of the gain comes from the questions that reward declining, and random vectors gain on those questions too. We have weakened the Llama base claim in the abstract, the contributions and Section 7. The calibrated v_truth shows no significant change on the questions that do not reward declining on any model. Tables 14 and 15 in the appendix have these values with intervals.

4. Novelty and scope. We agree that each control exists in prior work. The related-work section now states more precisely what is new: these directions are tested against random vectors matched to each direction's own layer and length across model families, and this revision adds two controls that the cited audits do not use, random directions drawn from the activation statistics and a design that separates the contrast from the position at which it is read. Both changed our own conclusions, which we take as evidence that they add information. We do not test SAE-based or probe-based directions, and the limitations now say so plainly.

5. The MC1 cost of the stronger context direction now has matched controls. We scored MC1 under the same 25 random vectors that are matched to the stronger direction (Table 21). MC1 falls significantly on two models. On Mistral, v_context lowers MC1 by 0.10, and the largest drop under a random vector is about a third of that. On Llama Instruct, v_faithful lowers MC1 by 0.17, and the largest random drop is less than half of that. The cost belongs to the direction. On the other three models the stronger direction does not change MC1 significantly (Table 2).

I thank the authors for their revision of the paper and additional experiments, my concerns are mostly addressed. I have increased my score accordingly.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response - new experiments and changes

We thank the three reviewers. Their main concerns point the same way: the random-vector controls were thin outside one model and weak in kind, the truthfulness metric could reward declining, and the two context directions could not be compared cleanly. We ran new experiments on each point. All changes are marked in colour in the revised PDF, and the run summaries are in the supplementary material.

New experiments

  1. Random vectors for every model: 25 isotropic vectors for each calibrated direction, scored on the same test items as the learned directions. These come in addition to the two 50-vector families the submission had for Llama Instruct.
  2. Structured random vectors, drawn from the covariance of the unsteered hidden states and from its top principal components (Reviewer dGfU).
  3. An abstention control matched to v_context on Mistral (Reviewer mhbd).
  4. A position control: all four combinations of the two contrasts and the two reading positions, at shared layers and doses, on four models (Reviewer 1HE5).
  5. For the truth direction: the standard single-answer MC1, a split by whether declining counts as correct, and settings chosen on validation questions (Reviewers mhbd and dGfU).
  6. The parts of the context score under the stronger direction of every model, next to its matched random vectors, and a judge label for coherent answers (Reviewer 1HE5).
  7. MC1 under the matched random vectors of the stronger context direction (Reviewer mhbd).

What changed in our conclusions

Three claims became weaker, and we rewrote them.

  • The stronger context direction of each model exceeds isotropic random vectors by a clear margin on all five models. Against structured random vectors it still ranks first, but on four models the best structured vector comes within 4 points, and on three of these a paired test on the same items cannot separate the two. The abstract now reports how close they come.
  • On Llama Instruct, most of the gap between the two context constructions comes from the layer that calibration chose, not from the contrast.
  • The later-layer truth result holds on Llama Instruct under every check. On Llama base most of the gain comes from questions where declining counts as correct.

We think these corrections strengthen the paper's argument. Each stricter control removed part of what a weaker control would have credited to the learned direction, and the paper is now organised around that sequence.

Presentation. We also rewrote the results paragraphs of Sections 4 to 7 so that each states its finding in words and carries few numbers. The rewriting changed no result. Sentences cite the table or figure that holds their values, and values that had been only in the text are in new appendix tables.

Two new checks support the remaining context claims. Under the stronger context direction, the judge labels more answers coherent on all five models. On the two models where that direction lowers MC1, Llama Instruct and Mistral, every matched random vector loses less than half as much, so the cost belongs to the direction.

Update: the runs that were in progress have finished, and the revised PDF, the supplementary material and the abstract are updated. No table, figure or appendix number has changed, and the main text still ends on page 10. The changes are listed below; the first one corrects a sentence of our reply.

  • Abstention on Mistral (Reviewer mhbd, point 1): a correction. Our reply said that the drop is specific to the direction. In a separate run on Mistral, with 50 random vectors per family at the layer and length of v_context, no isotropic vector and no vector in the top 64 principal components lowers detected abstention as far as v_context, but 4 of 50 vectors drawn from a hidden-state covariance do. The drop is therefore specific to the direction against isotropic random vectors, but not against all structured ones. This matches Section 4, where structured random vectors are the far stronger control. Section 5 now says "isotropic", and the Limitations and Appendix K report this run; its summary is in the supplementary material.

  • Abstention for Qwen 3.5 (Reviewer 1HE5, W2). Tables 2 and 22 now use ten random vectors for Qwen 3.5, as for the other models; our reply said three. The conclusion does not change. Under v_faithful the abstention rate lies just below the range of the ten random vectors, and the change is not distinguishable from zero.

  • Position control on Qwen 3.5 (Reviewer 1HE5, W1). Table 18 now includes Qwen 3.5, so the control covers all five models; our reply said four. On Qwen 3.5 neither v_context nor v_faithful is consistently ahead at a shared layer, and each contrast is weaker when it is read at the other position, so the reading position matters on this model too. On Qwen 3.5 the two submitted directions repeat the earlier equal-dose run within 1 point, not exactly as on the other models; Appendix J says so.

  • MC1 cost of the stronger context direction (Reviewer mhbd, point 5). Table 21 now covers all five models, with five random vectors for Qwen 3.5, the largest model, and 25 for the others. On Qwen 3.5, as on Llama base and Qwen 2.5, the MC1 change lies inside the range of the matched random vectors. As before, MC1 falls significantly only on Llama Instruct and Mistral, and there every matched random vector loses less than half as much.

  • Later-layer truth direction under the standard MC1 (Reviewers mhbd and dGfU). Table 15 now also scores the validation-selected setting on the standard single-answer MC1, next to the best matched random vector on that format. Two results are new.

    • On Mistral this setting gives a small gain on the standard MC1, larger than under any of the ten random vectors, while the MC1 of the main text does not change significantly. The abstract, the contributions and Section 7 now say that the three models other than the Llama models show little or no gain, not that they show none. In our reply to Reviewer dGfU, "the other models show none" refers to the setting of Table 4, and it still holds there.
    • On Llama base, validation selects layer 12. There the gain is the same on both formats and holds on the questions that do not reward declining. At that layer, however, most of the model's answers under a false passage match neither answer (note under Table 4), so the main text reports layer 18. Our statements about Llama base, that the gain roughly halves on the standard format and comes mostly from questions that reward declining, refer to layer 18. Section 7 and Appendix I now name the setting.
  • Structured random vectors with the learned component removed (Reviewer dGfU, point 1). On Mistral and on Llama Instruct we removed from each of the 40 structured vectors its component along the learned direction and kept its length (four new rows in Table 7; Appendix B). On Mistral, where we reported that the most effective vectors are partly aligned with the learned direction, they keep most of their gain, and the best of them still comes within 5 points of the learned direction. The alignment therefore accounts for the smaller part of their effect. On Llama Instruct, where the best vectors were already almost orthogonal to it, they keep their gain. This supports the reading in Section 4: a push of this length along a direction in which the hidden states vary strongly reproduces most of the effect by itself.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.