acceptodds
Under review as a conference paper at ICLR 2027

Steering False Event Reports: Fabricated Support and Subsequent Evidence Use

Abstract

Activation steering induces false event reports with fabricated support. In 96 blinded generations from 16 events, two independent raters find fabricated support in 0.72 and 0.69 of steered responses, versus at most 0.06 in controls (\kappa=0.667). Present-minus-absent residual directions raise false-report probability by 0.726–0.999 over norm-matched random controls across three Qwen2.5 checkpoints; unrelated-event directions have equivalent graded effects. These experiments link steered reports to generated supporting content in the tested settings.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, language models, causal interventions, event reporting, confabulation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jTrJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary
  • This paper uses activation steering to make language models say that an event happened when the given text does not mention it, and analyzes the resulting hallucinations.
  • The authors observe that false reports can come with fabricated supporting reasons, that a previous answer changes how much the model benefits from checking the record, and that the model's content choices can stay stable even when its answer changes.
Critical strengths
  • The paper includes many control experiments and analyses, such as norm-matched random vectors and vectors from other events.
  • It uses a controlled dataset with known labels.
Critical weaknesses
  • The paper is very hard to read.
  • Many terms are not explained. For example, "present-minus-absent residual direction" appears in the abstract without explanation and is only defined in Section 3.2.
  • The setting is not natural, and the motivation is unclear. The paper does not explain how these findings can be used or how they would help future model development.
  • Vectors from other events (Section 4.3, Figure 2) and from unrelated word pairs (Section 4.10, Figure 4) have almost the same effect. This suggests the intervention may not change the model's answer about the event itself, but simply make it more likely to say "Yes."
  • The human evaluation is small (two raters, 16 events), and agreement between raters drops in the additional audit (κ = 0.667 → 0.494).
Questions for authors
  1. In what real-world situation would a model's internal state be shifted in this way? How can these findings be used?
  2. If many different directions have the same effect, what does the intervention actually change?
  3. Appendix A.1 shows that giving a "Yes" draft without steering gives the same two-pass result. What does steering add to this experiment?
Reviewer_LQpYOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper describes the behavior of small models when they are steered toward an incorrect answer to a question based on provided context. Specifically, it finds that when steering is applied that encourages the model to affirm an event that doesn't occur in some context, the models affirm it more often than they did without steering. Furthermore, when asked to justify the incorrect answer, the model fabricates an explanation for the incorrect answer with low recognition of the inaccuracy. This occurs even when the steering direction is not specific to an event and cannot be easily corrected, even when running an unstirred second pass on a draft answer to the question.

Critical strengths

The task being studied was designed in a straightforward way and tested iteratively and extensively through several related experiments. This includes several baselines, ablations that cover different models and layers, and statistical testing. I appreciate the care provided through these components and the attention to transparency about the robustness of results.

Critical weaknesses

I found the paper quite hard to follow. The motivation of the problem, as well as the methods, are quite underspecified, and in some cases seem to focus on details that are less relevant. This made it challenging to contextualize and interpret results. My primary takeaways from the study were that steering in the direction of an incorrect answer consistently causes more incorrect responses as well as incorrect justifications for those answers, and that even when steering is dropped after the first attempt at an answer, the inaccurate response persists. The first part of this seems quite similar to Cox et al.'s paper cited by the authors. And in general, the differences that are illustrated seem small.

Questions for authors
  • What does this paper establish that prior work on pre-committed answers does not? How does the pre-committed effect compare to the fabricated effect in a similar setting?
  • What real-world situation might these results and experiments be relevant to?
  • What happens when answering that are unrelated to event reporting, but still have a "no" answer? e.g. "Is France in Asia?" Wondering if the effect generalizes.
Reviewer_dEQbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper applies activation steering to reports about fictional event records. It measures false answers, fabricated supporting text, and whether an earlier answer changes how the model uses a record during a later verification step.

Critical strengths

The distinction between changing an answer and changing its supporting content is worth studying. The unrelated-event, affirmative-direction, and label controls help identify limitations of a simple interpretation of the steering effect.

Critical weaknesses
  1. The presentation makes the scientific contribution unnecessarily difficult to assess. The abstract reports effect sizes before explaining the event-reporting task or what counts as fabricated support. The main text then moves between binary reports, content choices, generated explanations, graded scores, and later verification, often with different intervention settings. Although definitions and a protocol map are provided, the paper does not make the relationship between these experiments clear enough. A concrete running example and a direct mapping from each research question to its evidence would substantially improve the argument.

  2. After disentangling the experiments, the main new insight remains uncertain. The affirmative-direction and crossed-label controls leave an important response-format interpretation. Meanwhile, the later verification experiment receives a binary draft, not the fabricated supporting text studied in the human evaluation. It therefore does not show that fabricated support causes poorer evidence use. The paper needs to distinguish these separate observations and explain what they jointly add to existing work on steered answers and post-hoc rationalization.

Overall, I recommend rejection because the presentation obscures the contribution and the experiments do not establish a sufficiently clear connection between the two main findings.

Questions for authors

What is the central conclusion that holds across the different readouts under comparable intervention settings? Is there evidence that fabricated explanations affect verification beyond the effect of a bare incorrect answer?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.