acceptodds
Under review as a conference paper at ICLR 2027

Do Large Language Models Learn Order-Dependent Rules?

Abstract

Can large language models (LLMs) capture sequential structure when outcomes depend on order? Theoretically and experimentally, we prove that ordered-compliance inherently leaks information through frequency patterns, creating predictive shortcuts that allow models to succeed without truly capturing causal sequential structure. To isolate true order learning, we introduce a shortcut-auditing protocol that quantifies the predictive power of non-sequential features (e.g., k-grams, pair counts) under label noise. We develop a controlled, semantic-free sequence framework with three mechanisms over a hidden set of key letters: ordered compliance, driven by key order at a fixed lag; global-property compliance, driven by the parity of the total key count; and key-inversion parity, driven by the parity of key-order inversions. Empirically, encoder and recurrent models reach the theoretical optimum (measured by AUC and F1) on ordered-compliance more efficiently than fine-tuned decoder-only LLMs. On global-property compliance, the tested sequence models remain near chance under standard training unless explicitly guided on feature relevance. On key-inversion parity, recurrent and transformer models can solve the task with four keys, while all tested models remain near chance with six. Our results demonstrate that high-quality performance on order-sensitive tasks often masks shortcut exploitation.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Sequence modeling, shortcut learning, model evaluation, temporal dependencies, synthetic benchmarks, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tjhvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether strong predictive performance on order-dependent sequence tasks demonstrates that a model has learned the underlying ordering rule. It introduces three synthetic mechanisms over a hidden set of key letters: ordered compliance (OC), key-inversion parity (KIP), and global-property compliance (GPC). The theoretical analysis characterizes available shortcuts and reference performance under label noise. The experiments compare statistical baselines, recurrent networks, transformers, and pretrained language models, with additional feature-revelation, shuffling, curriculum, and clinical analyses. The paper reports that simple features can approach optimal OC performance, some models solve four-key KIP but fail on six-key KIP, and standard training generally fails on GPC unless task-relevant information or an appropriate curriculum is supplied. The main proposed contribution is an auditing framework that separates predictive success from evidence of learning the intended sequence mechanism.

Critical strengths
  1. A useful distinction between predictive performance and learning the intended rule. The paper addresses an important evaluation problem: a model can predict an order-dependent outcome using correlated statistics without recovering the full generating mechanism. The combination of controlled synthetic rules, feature-based baselines, and noise-aware oracle references offers a potentially useful way to examine this distinction. Removing semantic content also helps isolate the role of the sequence structure.

  2. Complementary diagnostic tasks and interventions. OC, KIP, and GPC expose different combinations of order dependence, hidden feature relevance, and parity computation. The shuffling controls, explicit key-order or membership inputs, and length curriculum are potentially informative for distinguishing representation discovery from subsequent computation. Together with the clinical example, these analyses could make the framework useful beyond a single synthetic benchmark if their empirical support and implementation are made consistent.

Critical weaknesses
  1. The experimental results are insufficiently supported and do not fully establish the claimed contributions.

    Table 5 mixes reported values with simulated completions without identifying measured entries, yet reports means and standard deviations. The repository's docs/MANUSCRIPT_GAPS.md states that no fine-tuned neural GPC metrics are archived and no completed 70B result is available. The paper also claims KIP-4 success for 70B while restricting its evaluation to GPC elsewhere. These inconsistencies undermine the GPC and model-scale comparisons.

    Table 4 reports 400K, three-run KIP results, but the released Llama-8B and Qwen-14B/27B sweep contains single-seed measurements through 200K and skips 300K/400K. The decoder-only KIP-4 success claim also conflicts with the LSTM results.

    Baseline matching establishes that shortcuts can support prediction, not that neural models use them. Shuffling and feature-revelation controls do not identify the learned algorithm. Although the introduction acknowledges this limitation, actual shortcut reliance across the tested models remains unestablished.

  2. The released implementation differs substantially from the experimental protocol in the paper.

    The paper specifies a joint language-modeling/classification loss and a two-layer head. Although the legacy script implements this design, the newer KIP pipeline uses classification loss alone and a single linear head. Its token limit and default batch settings also differ from Table 1, and archived RoBERTa fits use base rather than the large model specified in the main methods.

    The KIP sweep uses --stop-on-perfect-test, stopping when test AUC reaches 1 after a qualifying validation checkpoint. This contradicts the validation-only stopping description and makes the reported performance and data threshold dependent on test-set selection.

    Equation 4 labels odd KIP inversion counts positive, whereas the released data label even counts positive. Appendix D.4 describes OC-Noisy rules, but phase4_heldout_rule.py generates noiseless pair-existence labels without the maximal-chain condition. These differences in objectives, models, labels, and evaluation prevent a reliable connection between the implementation and the reported experiments.

Questions for authors
  1. Which neural-model entries in Table 5 are based on completed experiments? The phrase "explicitly simulated completions" is undefined, and the table does not identify the entries to which it applies.

  2. Which per-seed runs support the 400K KIP results in Table 4? The released Llama-8B and Qwen-14B/27B sweep contains only one seed through 200K and skips 300K/400K.

  3. Which reported results used test-set AUC as a stopping criterion? The released KIP trainer includes --stop-on-perfect-test, whereas Table 1 specifies stopping based on validation loss.

  4. Which labeling rule generated the data for Appendix D.4? The paper refers to the OC-Noisy family with a maximal-chain condition, whereas phase4_heldout_rule.py generates noiseless pair-existence labels.

  5. Which training configuration produced each main pretrained-model result? The paper and released code differ in loss function, classification head, token limit, batch settings, and RoBERTa model size.

Reviewer_MdneOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper explores whether large language models truly learn order-dependent rules or merely exploit statistical frequency patterns, called predictive shortcuts. To examine this, the authors propose three controlled, semantic-free sequence evaluation tests: (1) Ordered Compliance (OC), (2) Key-Inversion Parity (KIP), and (3) Global-Property Compliance (GPC). Through experiments testing various LLMs, transformers-based sequential models, XGBoost, and RNNs, the authors empirically and theoretically argue that models tend to achieve high performance by exploiting these predictive shortcuts rather than capturing the actual causal sequential structures.

Critical strengths
  • The approach of three systematic, semantic-free frameworks (OC, KIP, GPC) to evaluate the sequence-understanding capabilities of language models, supported by mathematical and theoretical proofs, is an interesting and reasonable direction of contribution.

  • It is useful that the authors did not limit their evaluation to controlled synthetic data but also applied their theoretical findings to real-world sequential prediction problems using clinical data (ie, Chronic kidney disease progression).

Critical weaknesses
  • The "predictive shortcut" issue highlighted in the paper may be interpreted as an inevitable overfitting problem that occurs when fine-tuning massive models on small datasets. It is difficult to definitively judge whether LLMs, which are pre-trained on massive datasets, fail to learn order-dependent rules based solely on fine-tuning experiments with relatively limited data. If the training data is scaled up, this overfitting issue might naturally be resolved. Thus, the provided experimental results alone seem insufficient to support the ambitious conclusion posed by the paper: i.e. "Do large language models learn order-dependent rules?".

  • Overall, the paper is difficult to read and follow. Specifically, the mathematical and theoretical definitions of the core evaluation mechanisms (OC, KIP, GPC) are complex, making it hard to grasp the concepts. To improve readability, the authors should provide more intuitive, high-level explanations or clear working examples before diving into the formula-heavy descriptions.

  • The result analysis section primarily lists numerical values (like AUC) with superficial descriptions, making it difficult to understand the deeper implications of these metrics regarding the models' internal workings. Furthermore, despite the extensive experiments conducted, there is only a single visual figure (Figure 1) illustrating the overall trends, which is insufficient for supporting comparisons across the various settings. The paper needs deeper qualitative and quantitative analyses, along with more diverse visualizations.

Questions for authors

See Critical Weaknesses

Reviewer_cFmdOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a worthwhile question: when labels are genuinely generated by the order of events, does strong performance by a sequence model (and by LLMs in particular) imply that the model has learned the order rule?

The authors construct three semantics-free mechanisms over letter sequences (n=20, ℓ=26, hidden key set S):

  1. OC (ordered compliance) — the label is 1 iff the sequence contains a maximal λ-run of keys that is non-decreasing under the hidden ranking κ;
  2. KIP (key-inversion parity) — each key occurs exactly once; the label is the parity of the number of inversions in the key ranking;
  3. GPC (global-property compliance) — the label is 1 iff the sum of key-membership bits B_t agrees with m mod 2; this mechanism is order-independent.

On the theory side, the paper establishes a "local purity" hierarchy: OC is not k-locally pure for any k (Thm 3.4 — even a single position leaks); KIP is exactly (m−2)-locally pure with tightness at m−1 (Thm 3.6); balanced GPC (p=1/2) is k-locally pure for every k<n (Thm 3.8).

Empirically, the paper compares BERT/RoBERTa, Llama-3.2-1B/3.1-8B/3.1-70B, Qwen3-4B-Think/2.5-14B/3.8-27B, randomly initialized LSTM / Transformer / BiLSTM-Transformer, XGBoost, and k-gram / pair-count baselines, and adds a real-data audit on MIMIC-IV CKD→ESRD prediction.

Critical strengths
  1. The problem is clearly framed.
  2. The count-preserving perturbation control on OC is well designed. On 192 oracle-verified permutations that preserve letter counts but flip the label, all four deterministic-OC models correctly flip their predictions; on 200 distractor-only swaps, none of them flips. This is a well-constructed sufficiency control.
Critical weaknesses
  1. The decisive shortcut-vs-rule criterion is tested only on the linear baseline, never on the neural models.
  2. The task suite has too little dynamic range, so the architecture conclusions effectively rest on a single task.
Questions for authors
  1. Could the leave-one-rule-out protocol of D.4 be applied to the neural models (BERT / LSTM / Llama, ≥20 random rules)?
  2. Why is Llama-3.1-70B evaluated only on GPC?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.