One Story, Two Orders: Time Overrides Text in LLMs

A story has two orders: the chronological order of its events and the text order in which they are mentioned. Chronological order can bias human memory for the order of presentation, but whether and how LLMs exhibit similar interference remains unclear. We independently manipulate text order and chronological order in controlled narratives across ten different LLMs. When the orders conflict, text-order accuracy drops by $63.3$ percentage points on average, whereas chronological-order judgments are substantially more robust. Yet both ranks remain linearly decodable from pre-query hidden states, with nearly perfect text-order rank decoding at the final layer. Causal interventions trace part of this interference to content encoding chronological order in an unchanged carrier region before the question, with larger effects from values than keys. Inspired by this mechanistic understanding, we propose ReFrame (Reference-Frame Routing Adaptation), adapting only attention query and value projections through low-rank updates. The method raises mean text-order conflict accuracy to $99.9\%$ on held-out original questions, improves chronological-order judgments, and extends these gains to additional question wordings. These findings expose a gap between what LLMs retain and what governs their answers. They identify selective use of preserved information as a distinct target for evaluating and improving reliability when chronological order and text order conflict.