- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether strong predictive performance on order-dependent sequence tasks demonstrates that a model has learned the underlying ordering rule. It introduces three synthetic mechanisms over a hidden set of key letters: ordered compliance (OC), key-inversion parity (KIP), and global-property compliance (GPC). The theoretical analysis characterizes available shortcuts and reference performance under label noise. The experiments compare statistical baselines, recurrent networks, transformers, and pretrained language models, with additional feature-revelation, shuffling, curriculum, and clinical analyses. The paper reports that simple features can approach optimal OC performance, some models solve four-key KIP but fail on six-key KIP, and standard training generally fails on GPC unless task-relevant information or an appropriate curriculum is supplied. The main proposed contribution is an auditing framework that separates predictive success from evidence of learning the intended sequence mechanism.
Critical strengths
-
A useful distinction between predictive performance and learning the intended rule. The paper addresses an important evaluation problem: a model can predict an order-dependent outcome using correlated statistics without recovering the full generating mechanism. The combination of controlled synthetic rules, feature-based baselines, and noise-aware oracle references offers a potentially useful way to examine this distinction. Removing semantic content also helps isolate the role of the sequence structure.
-
Complementary diagnostic tasks and interventions. OC, KIP, and GPC expose different combinations of order dependence, hidden feature relevance, and parity computation. The shuffling controls, explicit key-order or membership inputs, and length curriculum are potentially informative for distinguishing representation discovery from subsequent computation. Together with the clinical example, these analyses could make the framework useful beyond a single synthetic benchmark if their empirical support and implementation are made consistent.
Critical weaknesses
-
The experimental results are insufficiently supported and do not fully establish the claimed contributions.
Table 5 mixes reported values with simulated completions without identifying measured entries, yet reports means and standard deviations. The repository's
docs/MANUSCRIPT_GAPS.mdstates that no fine-tuned neural GPC metrics are archived and no completed 70B result is available. The paper also claims KIP-4 success for 70B while restricting its evaluation to GPC elsewhere. These inconsistencies undermine the GPC and model-scale comparisons.Table 4 reports 400K, three-run KIP results, but the released Llama-8B and Qwen-14B/27B sweep contains single-seed measurements through 200K and skips 300K/400K. The decoder-only KIP-4 success claim also conflicts with the LSTM results.
Baseline matching establishes that shortcuts can support prediction, not that neural models use them. Shuffling and feature-revelation controls do not identify the learned algorithm. Although the introduction acknowledges this limitation, actual shortcut reliance across the tested models remains unestablished.
-
The released implementation differs substantially from the experimental protocol in the paper.
The paper specifies a joint language-modeling/classification loss and a two-layer head. Although the legacy script implements this design, the newer KIP pipeline uses classification loss alone and a single linear head. Its token limit and default batch settings also differ from Table 1, and archived RoBERTa fits use base rather than the large model specified in the main methods.
The KIP sweep uses
--stop-on-perfect-test, stopping when test AUC reaches 1 after a qualifying validation checkpoint. This contradicts the validation-only stopping description and makes the reported performance and data threshold dependent on test-set selection.Equation 4 labels odd KIP inversion counts positive, whereas the released data label even counts positive. Appendix D.4 describes OC-Noisy rules, but
phase4_heldout_rule.pygenerates noiseless pair-existence labels without the maximal-chain condition. These differences in objectives, models, labels, and evaluation prevent a reliable connection between the implementation and the reported experiments.
Questions for authors
-
Which neural-model entries in Table 5 are based on completed experiments? The phrase "explicitly simulated completions" is undefined, and the table does not identify the entries to which it applies.
-
Which per-seed runs support the 400K KIP results in Table 4? The released Llama-8B and Qwen-14B/27B sweep contains only one seed through 200K and skips 300K/400K.
-
Which reported results used test-set AUC as a stopping criterion? The released KIP trainer includes
--stop-on-perfect-test, whereas Table 1 specifies stopping based on validation loss. -
Which labeling rule generated the data for Appendix D.4? The paper refers to the OC-Noisy family with a maximal-chain condition, whereas
phase4_heldout_rule.pygenerates noiseless pair-existence labels. -
Which training configuration produced each main pretrained-model result? The paper and released code differ in loss function, classification head, token limit, batch settings, and RoBERTa model size.