False Witnesses: When Irrelevant Facts Change Transformer Answers

We identify and study a failure mode in which a true relation about the wrong entity (*irrelevant fact*) steers a transformer toward a wrong answer. Specifically, we study a controlled two-step reasoning task that composes independent random functions: the first maps the queried entity to an intermediate value, and the second maps this value to the final answer. This defines *relevance* exactly, avoiding the interpretive ambiguity of natural-language contexts. An optional suggestion in the input prompt proposes the intermediate value. We call a *true* first-step relation about *another* entity that matches an *incorrect suggestion* a **false witness**. Our **equal-function test** changes whether this relation is present while preserving every entity's correct final answer, isolating its effect on suggestion following. Across two-layer transformers trained from different seeds, adding a false witness doubles predictions following the incorrect suggestion, from 27.4% to 54.8%. Causal interventions via activation patching identify the underlying attention mechanism: query–key matching ($Q/K$) favors the irrelevant entity and its value vector ($V$) supplies the answer. Replacing $Q$ alone can recover the queried entity's answer. Training dynamics reveal a gap between solving the task and resisting misleading suggestions: models first become accurate only without suggestions, while resistance to incorrect suggestions develops later. This gap closes when each entity can look up its own final answer in the first attention layer; accuracy under incorrect suggestions then rises from 38.3% to 98.9%, nearly matching accuracy obtained without suggestions. In sum, we establish *false witnesses* as a failure of *relevance*, identify the attention mechanism responsible, and show how training and information access shape resistance to false witnesses.