acceptodds
Under review as a conference paper at ICLR 2027

One Story, Two Orders: Time Overrides Text in LLMs

Abstract

A story has two orders: the chronological order of its events and the text order in which they are mentioned. Chronological order can bias human memory for the order of presentation, but whether and how LLMs exhibit similar interference remains unclear. We independently manipulate text order and chronological order in controlled narratives across ten different LLMs. When the orders conflict, text-order accuracy drops by percentage points on average, whereas chronological-order judgments are substantially more robust. Yet both ranks remain linearly decodable from pre-query hidden states, with nearly perfect text-order rank decoding at the final layer. Causal interventions trace part of this interference to content encoding chronological order in an unchanged carrier region before the question, with larger effects from values than keys. Inspired by this mechanistic understanding, we propose ReFrame (Reference-Frame Routing Adaptation), adapting only attention query and value projections through low-rank updates. The method raises mean text-order conflict accuracy to on held-out original questions, improves chronological-order judgments, and extends these gains to additional question wordings. These findings expose a gap between what LLMs retain and what governs their answers. They identify selective use of preserved information as a distinct target for evaluating and improving reliability when chronological order and text order conflict.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Temporal Reasoning, Reference-Frame Competition, Event Chronology

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_f8aPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies conflicts between the order in which events are presented and the chronological order in which they occur. Using synthetic four-event narratives evaluated on ten language models, the authors report that chronological order strongly disrupts text-order questions, while chronological-order questions are more robust. Linear probes recover both orderings from hidden states, and cache transplantation and access-mask interventions are used to argue that chronology-conditioned contextual states causally influence text-order answers. The paper then proposes ReFrame, a low-rank adaptation of attention query and value projections, which achieves near-perfect accuracy on held-out instances of the same controlled task and improves performance under several alternative question wordings. The empirical effects are interesting, but the paper’s mechanistic claims and the generality of the proposed adaptation are not sufficiently established.

Critical strengths
  1. The paper identifies a clear and reproducible behavioral asymmetry: chronological order is substantially more likely to override text-order answers than vice versa. The factorial construction and evaluation across ten models make this a useful diagnostic phenomenon.

  2. The authors provide more mechanistic analysis than a purely behavioral paper, including cross-model probing, carrier transplantation, K/V separation, and access controls. These experiments offer evidence that contextual states formed during passage processing influence later order judgments.

Critical weaknesses
  1. The benchmark does not convincingly distinguish chronological-order interference from a much simpler instruction- following or positional heuristic failure. Text-order questions explicitly instruct the model to ignore dates, while chronological questions explicitly instruct it to ignore sentence order. The conflicting conditions therefore change not only the underlying order relation but also which salient cues must be suppressed. A model that defaults to dates, recency, or semantic temporal structure can produce the reported asymmetry without constructing two competing internal order representations. The paper needs stronger controls, such as non-temporal distractor attributes, randomized date formats, decoy numerical information, and conditions where text order is queried without explicit “ignore dates” instructions.

  2. The causal interventions do not establish the claimed mechanism with sufficient specificity. Carrier transplantation replaces keys and values at every full-attention layer, producing hybrid activations that are not naturally generated by the recipient prefix. The carrier is also not an isolated memory slot: its contextual state depends on the entire preceding passage and can encode many features besides chronology. Similarly, context access masks affect broad token ranges and, for Qwen3.5, recurrent state updates. Thus, the results show that these hidden states can influence the answer, but not that chronological information is routed through a distinct carrier pathway or that values specifically carry the proposed order representation.

  3. The probing evidence is weaker than the paper suggests. The probes operate on complete sentence-end hidden states with positional information intact. Near-perfect prediction of text rank may therefore be explained by absolute or relative position signals, without demonstrating a content-based representation of “mention order.” The paper acknowledges this limitation, but still uses the decoding result as central evidence for a gap between “retaining” and “using” text-order information. A position-ablated, content-matched, or amnesic probing analysis is needed.

  4. The ReFrame evaluation is too close to the training problem to support the broad claims. ReFrame is trained on the same four-event synthetic task with shared templates, answer format, instruction structure, and controlled order manipulations. The “additional questions” change wording but preserve the same scenes, dates, options, and underlying task. Near-perfect conflict accuracy can therefore reflect task-specific supervised correction rather than a general mechanism for selecting reference frames in narrative understanding. The paper does not evaluate on genuinely different stories, longer narratives, natural flashbacks, unseen temporal expressions, or external temporal-order benchmarks.

  5. The reported mechanistic explanation and the proposed method are not tightly linked. The interventions identify effects of contextual states, but ReFrame simply adapts Q and V projections across the model. The paper does not show that ReFrame changes the specific carrier pathway identified by transplantation, nor that the relevant causal effect is reduced through the proposed mechanism. The method may instead learn a direct benchmark-specific decision rule or strengthen instruction-following. Without pathway-level validation after adaptation, the claim that ReFrame is mechanism-inspired in a substantive sense remains largely rhetorical.

Questions for authors
  1. How do the authors rule out the simplest explanation that the models rely on salient timestamp information or a chronological default, rather than representing two competing order coordinates? Please provide controls with randomized or non-date temporal expressions, irrelevant numerical distractors, and matched “ignore X” instructions.

  2. Can the authors repeat the probing analysis after removing or randomizing positional information, or use an amnesic intervention that tests whether position information is causally necessary for text-order decoding? Without this, what evidence shows that the probe is decoding text-order content rather than position?

  3. What is the quantitative difference between natural recipient states and the donor-transplanted hybrid states? Have the authors verified that the intervention remains on-manifold, for example by comparing hidden-state distributions, likelihoods, or downstream behavior under progressively smaller interventions?

  4. Why should the “carrier” be interpreted as a specific chronological-information route rather than an arbitrary late-context bottleneck? Can the authors perform matched transfers from multiple equally sized regions, including earlier and non-timestamp regions, with controls matched for norm, attention mass, and state similarity?

  5. Does ReFrame improve performance on genuinely unseen narrative structures, event descriptions, temporal expressions, and longer stories? Evaluation on only new question wordings over the same scenes is not sufficient to establish generalization.

  6. Can the authors compare ReFrame against simpler baselines, such as supervised fine-tuning on the same examples, LoRA on MLP/output projections, prompt-level instruction improvements, or training only query/value layers without the consistency objective? This is necessary to determine whether the reported gains come from the proposed mechanistic design or simply from task-specific adaptation.

  7. After ReFrame training, do the carrier-transplantation and K/V intervention effects change in the predicted direction? If not, what evidence supports the claim that ReFrame repairs the mechanism identified earlier?

Reviewer_VijLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how chronological order interferes with text-order reasoning in LLMs. Through experiments across ten models, the authors show that chronological information can override text-order judgments even when the correct positional information remains accessible in hidden states. Based on causal interventions, they propose ReFrame, a lightweight Q/V LoRA adaptation method that significantly improves order-related reasoning accuracy.

Critical strengths
  1. The paper identifies an interesting and consistent failure mode across ten LLMs.
  2. The proposed ReFrame method achieves substantial improvements across multiple models, demonstrating the practical potential of addressing order interference.
Critical weaknesses
  1. Insufficient ablation studies. ReFrame combines Q/V adaptation, supervised training, and consistency regularization. However, the paper lacks comparisons with standard LoRA, Q-only/V-only adaptation, and training without consistency regularization.
  2. Limited generalization. The experiments primarily use short synthetic narratives containing four events and constrained A/B answers. Although additional question wordings are evaluated, generalization to longer narratives, realistic documents, and open-ended reasoning remains unverified.
Questions for authors
  1. Can the authors compare ReFrame against standard LoRA under matched parameter budgets and training conditions, including Q-only, V-only, and Q/V adaptations?
  2. How much does each component of the training objective contribute to the final performance?
  3. Does ReFrame generalize to longer narratives with more events and unconstrained response generation?
Reviewer_gtapOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the LLMs behavior on two orderings of a narrative: the text order (the order in which events are mentioned in the input) and the chronological order (the order in which events actually happened, signaled by explicit timestamps). The authors create a four-event series where sentence position and dates vary independently, and benchmarks ten LLMs on this data. The authors also introduce a new method -- Reframe, which raises text-order conflict accuracy to 99.9%, and improves chronological accuracy to 95.3%.

Critical strengths

The authors cross sentence swaps and date swaps independently, counterbalance answer options, and compare each task to its own congruent baseline. This separates interference from baseline difficulty and shows that models are actively answering the wrong question when the two orders disagree, not just struggling with harder inputs.

The text-order conflict cost appears in all ten models from seven families, including a hybrid full/linear-attention architecture, and under all three question wordings. Several models fall well below chance, so the effect reflects a systematic preference for chronology rather than noise.

Events, dates, and options are held fixed while four question pairs absent from training are crossed, so wording changes are isolated from content changes.

Critical weaknesses

The probes decode the absolute rank of sentence-end tokens, which is likely recoverable from position-correlated features. The authors acknowledge the probes “do not establish a position-independent code,” and there is no position-only baseline or probe at the answer position, where the decision is actually made.

Dates are explicit words while text order is implicit and unlabeled, so a model can answer congruent text-order questions just by reading dates. Table 10 supports this concern: the relevant text-order gain is tiny for Granite (0.017) and Yi (0.078) compared with irrelevant chronological gains of about 0.34 to 0.37. Without a condition with explicit indices (“Sentence 1: …”), “chronology overrides text order” cannot be distinguished from “explicit dates dominate an implicit cue.”

There is no comparison to K-only, V-only, Q-only, O/MLP, or full-projection LoRA at matched parameters, no fine-tuning without the consistency term, and no prompting baseline, so it is unclear what drives the gain. The authors also state that the behavioral results “do not directly establish changes to the same carrier pathway,” and carrier transplantation is never rerun on ReFrame models, so the “mechanism-inspired” framing is unsupported.

All models are 4 to 14B, evaluation is next-token A/B only, and there are no chain-of-thought or few-shot baselines.

Questions for authors

If you add explicit sentence numbers to the passage, does the asymmetry shrink or disappear? This would tell us whether the effect is about representation or just about dates being more visible.

Does chain-of-thought, few-shot examples with conflicts, or a larger model close the gap? If simple prompting fixes it, the significance of both the finding and ReFrame changes.

How does ReFrame compare with LoRA on keys only, values only, queries only, or MLPs at the same parameter count, and with the consistency and retention losses removed? Without this, it is unclear whether the Q and V choice matters.

If you prepend explicit indices ("Sentence 1: …"), does the text-vs-chronological asymmetry shrink or vanish? This is the decisive test between representational interference and date salience.

Can you localize the carrier effect to specific layers/heads and compare to the Pink et al. order-reinstating head?

Reviewer_uGN6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLMs can distinguish the order in which events are mentioned from the order in which they occur. The authors construct synthetic four-event passages in which sentence order and timestamps can be varied independently, and evaluate ten instruction-tuned models. They find that conflicts between the two orders substantially reduce text-order accuracy, while chronological-order accuracy is less affected.

The paper further investigates this asymmetry through linear probing and causal interventions on attention keys and values. Finally, it introduces ReFrame, a Q/V LoRA-based adaptation method that substantially improves performance on the original task and transfers to alternative question wordings.

The paper addresses an interesting question and presents a carefully designed behavioral and mechanistic analysis. However, some of its central interpretations remain insufficiently supported, particularly the claim that chronological information overrides an otherwise available text-order representation. The contribution of ReFrame is also unclear without stronger baselines and ablations.

Critical strengths

Clear research question and experimental design. The factorial design independently varies sentence order and timestamps, with counterbalanced answer options. This provides a clean way to study the interaction between text order and chronological order.

Broad and consistent evaluation. The authors test ten models from seven model families and find a consistent asymmetry across them. The reported per-model results support the paper's main behavioral observation.

Careful mechanistic analysis. The carrier-region interventions include several useful controls, such as self-transfer, sham transfer, blocked-question variants, and neutral-key masks. The authors also acknowledge that the measured effects are partial and non-additive.

Relevant problem and transparent reporting. The distinction between information encoded in hidden states and information used to produce an answer is important. The paper also reports uncertainty estimates and discusses several limitations of its experiments.

Critical weaknesses

W1. The override interpretation is not sufficiently established. The results do not rule out a simpler explanation: models may rely primarily on timestamps and make little use of text order in the first place. High accuracy on congruent trials does not establish text-order competence because timestamps provide the same answer. Indeed, the probing results show that chronological information can have a stronger influence than text-order information. The context-masking results provide some evidence for the override interpretation, but cover only two models and a limited set of trials. A baseline with timestamps removed or made uninformative, evaluated across all ten models, is necessary to distinguish interference from a lack of text-order competence.

W2. The probing evidence has important limitations. The near-perfect decoding of text-order rank from sentence-end states may simply reflect absolute sentence or token position, rather than an event-level representation that can be retrieved when answering a question. Decoding at the question or answer position would provide more direct evidence. The chronological-rank probes also risk exploiting a date-to-rank lookup: under causal attention, early sentence states cannot observe the dates of later events, and the train/test splits reuse the same timestamp quadruples. The authors should report results by text position and use disjoint timestamp sets across probe splits.

W3. ReFrame's methodological contribution and generalization remain unclear. ReFrame applies LoRA to query and value projections, which is already a standard adaptation configuration. Without comparisons against other LoRA placements at matched parameter counts, prompting, or simple sentence numbering, it is difficult to establish that the proposed design offers a meaningful advantage. The method also lacks component ablations, including Q-only and V-only adapters and the removal of the consistency and capability losses.

The evaluation further uses training and test passages generated from the same restricted setup. The results on alternative wordings are encouraging, but do not establish generalization to longer passages, more events, implicit or relative dates, or natural text. The capability checks are also insufficient to demonstrate retention: capability examples receive supervised training, native arithmetic accuracy is unexpectedly low, and no standard benchmark is reported. Finally, the reduction in angular deviation does not consistently correspond to a comparable reduction in irrelevant gains. This makes the claim of functional disentanglement stronger than the evidence warrants. The authors also acknowledge that ReFrame's effect on the proposed carrier pathway has not been established.

W4. The evaluation scope and statistical treatment limit the conclusions. The study covers only 4–14B models and evaluates answers through forced-choice next-token logits, without examining whether the effect persists with larger models or when reasoning is enabled. In addition, the 64 worlds reuse only 16 timestamp quadruples. Since the bootstrap treats worlds as independent, it may underestimate uncertainty for effects associated with timestamps. Resampling at the timestamp-quadruple level would provide a more appropriate estimate.

W5. The synthetic task is not validated against human interpretation or natural text. Although the dataset is carefully balanced, its diversity is limited: passages contain four events, dates appear at the beginning of every sentence, and event descriptions follow shared templates. There is no human evaluation confirming that participants interpret the questions as intended, particularly instructions such as “mentioned first” and “ignore the stated times.” Nor is the phenomenon tested on natural text involving flashbacks or relative timestamps. A small human baseline and a more natural evaluation set would help establish the validity and broader relevance of the findings.

Questions for authors

(1) How can the authors distinguish chronological override from a lack of text-order competence? Could they evaluate all models without informative timestamps?

(2) Do the probing results reflect genuine event-order representations rather than positional cues or timestamp memorization? Could the authors use disjoint timestamps and probe at question/answer positions?

(3) What is the methodological novelty of ReFrame compared with standard LoRA? Could the authors provide stronger baselines and component ablations?

(4) How well does ReFrame generalize to longer passages, more events, and natural narratives? Can capability preservation be verified using standard benchmarks?

(5) What evidence supports functional disentanglement, given the inconsistent relationship between angular deviation and irrelevant gains?

(6) Do the findings hold for larger models and free-form reasoning? Could the authors account for shared timestamps in their statistical analysis?

(7) Could the authors include human validation and natural-text experiments to establish the validity and broader relevance of the synthetic task?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.