acceptodds
Under review as a conference paper at ICLR 2027

Tokenization Invariance Is Learned Late and Broken at the Head

Abstract

Splitting the input of a subword language model into single characters raises its bits per character five- to thirteen-fold, which is commonly taken to mean that the model is tied to the tokenization it was trained on. We argue that the failure is largely confined to the output head, and that what the trunk has learned survives re-tokenization on a schedule and with a specificity that current accounts do not predict. Probing fifteen pretraining checkpoints of OLMo-2-7B with frozen linear probes and matched random-direction control probes, we find that the model solves a word-order task within its first 9 billion tokens, but that the probe direction becomes reliably readable on character-split input, after a per-layer mean correction, only at about 630 billion tokens, roughly seventy times later. This invariance is specific to the task direction, increases with scale within OLMo-2 (one anomalous final checkpoint is flagged), and has not appeared by the end of two shorter Pythia runs. In released models it is graded in the fraction of tokens split and specific to the construct: no semantic-content probe in four models transfers more selectively than random directions in a way that survives a change of pooling rule, so re-tokenization spares positional structure in strong trunks rather than content. On the generation side, every model continues character-split prefixes at 1.3 to 1.8 times its canonical bits per character but emits character-split text at 5 to 13 times, and restricting the output distribution to character tokens removes at most a quarter of that collapse. Retraining the output head, 6.5% of the parameters, over the frozen trunk brings Llama-3.1-8B to within 0.63 bits per character of canonical; a capacity-matched adaptation of the trunk, run as a registered control, repairs less and does not preserve canonical reading. The part of the head repair attributable to the trunk tracks probe-measured invariance across eight models (exact one-sided Spearman , , registered in advance). Twenty predictions were registered before their data were collected; twelve failed, and every outcome is reported.

Primary area:
interpretability and explainable AI
Keywords:
tokenization, language model, mechanistic interpretability, model representations

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_B36QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

When a subword language model is given character-split input, its performance collapses, which is usually taken to mean the model is tied to its tokenizer. This paper argues instead that the failure lies mainly in the output head, while the rest of the model still understands the input. Probing fifteen pretraining checkpoints of OLMo-2-7B with frozen linear probes and matched random-direction control probes, it finds that the model learns a word-order task by 9B tokens but only becomes robust to character splitting around 630B tokens, and that this robustness holds for word order but not semantic content. Retraining only the output head of Llama-3.1-8B over a frozen trunk restores generation performance to near-canonical levels, and the trunk’s contribution to that repair correlates with probe-measured invariance across eight models.

Critical strengths

One of the major strengths of the paper is its transparency. The paper preregistered 20 quantitative predictions and openly reports all outcomes (8 passed, 12 failed), giving a clear picture of the findings. The read/write asymmetry is also a useful finding. Models read character-split text at modest cost but cannot produce it, and a head retrained over the real trunk restores the performance to near-canonical while one over a random trunk cannot. It rests on direct intervention rather than probe interpretation. It gives tokenizer-transfer and byte-level conversion work a concrete target, and it is the main reason I see real value in the paper despite my concerns about the probing claims.

Critical weaknesses

The main claims are much broader than the experiments behind them. The 70× delay comes from one training run of one model (OLMo-2-7B) on one task (detecting shuffled English sentences). I don’t see how this supports a general statement about when models learn tokenization invariance. Would the same delay appear with a different positional task or in another model family with checkpoints past 500B tokens? The Pythia comparison doesn’t help here: both Pythia runs stop at 300B tokens, before OLMo-2-7B itself crossed the threshold (428B), so we can’t tell whether Pythia never gets there or just hasn't yet. Likewise, "content does not survive" is tested with a single NLI task, and "syntax" with BLiMP alone. These are claims about three datasets, not about constructs.

I also have some concerns about the trunk adaptation controls. To test whether adaptation is localized specifically to the head, the authors trained LoRA adapters on Llama-3.1-8B's trunk while keeping the head frozen. The trunk arm degraded canonical text reading from 1.31 to 6.73 bpc (Table 2, Row 20; Table 7). Because this control severely damaged standard reading capabilities, it leaves open whether trunk adaptation is inherently incapable of character generation or if the LoRA training setup caused catastrophic interference. Without a working control, the claim that the failure is localized to the head rests on the head experiment alone.

Questions for authors
  • Does the late-invariance finding hold for a second positional task, or for another model family with checkpoints beyond 500B tokens? Forward-pass mean translation patching removes at most 27% of the collapse at the best single layer. Why can't the unretrained output head process this shifted trunk geometry without explicit parameter updates? (Figure 8)

  • Trunk LoRA adaptation degraded canonical reading from 1.31 to 6.73 bpc. Did you test lower ranks (e.g., r=16) to confirm whether this degradation is an artifact of the high rank (r=192, 503M params) rather than a trunk limitation? (Table 7)

  • A 100% character head drops SST-2 accuracy (0.70 to 0.55) on character-split prompts, while a 50/50 mix preserves accuracy but leaves bpc elevated (2.80 vs. 1.63 baseline). Did you evaluate asymmetric mixing ratios (e.g., 90/10) to improve this trade-off?

  • Since trunk invariance holds for word-order shuffling but fails for semantic content (NLI) and syntax (BLiMP), does this invariance generalize to fine-grained character-level or symbolic tasks (e.g., spelling or arithmetic)?

Happy to discuss these questions during the rebuttal period, and I am open to reconsidering my scores if my concerns get addressed.

2 comments
AuthorsOpenReview ↗

Response to Reviewer B36Q

Thank you. We have carefully checked each of your points. We realise that there were some issues in our methodology and we have taken the opportunity to address them. We also realise that the paper was not in the best shape that it could have been and understand the rating. We have updated the PDF, but of course we do not expect a second review. We've summarised the changes below:

To address one run, one task and one family, we added Amber-7B and an adjacent-swap task with a registered dating rule requiring selectivity. None of the three added runs is acquired, so "seventy times" is removed and the late-transfer statement is restricted to OLMo-2. The only ratio available is the stored OLMo-2-7B one, 47.6, with fixed-trim controls. Pythia is described as censored.

We agree that three datasets do not establish constructs, and the paper now names tasks. We added Rotten Tomatoes sentiment, CoLA acceptability and typo detection. Sentiment clears under both pooling rules only in Llama-3.1-8B, CoLA and BLiMP clear under both in no model, and typo detection clears in no cell (updated Table 3).

For the LoRA that damaged canonical reading, and for lower ranks, we ran ranks 16, 64 and 192 at three learning rates, with a dense arm and a mixed-stream arm. A working control exists (W passes: 3 of 14 configurations). Rank 16 at 10−510^{-5} reaches 1.54 bpc at canonical 1.49, better than head-only (1.94), so the head has no advantage in this comparison and we remove the comparative sentence.

The unretrained head cannot use the shifted geometry (Fig. 8) because the mean shift fixes only the first moment. We tested this at the head input directly. A mean shift reaches 16.19 bpc, while a full affine map and a rank-64 map reach 1.78, against 1.77 for the retrained head (Y6 fails). A map at the head input is enough to repair writing.

For mixing ratios, character shares from 0.1 to 0.9 give character-split bpc of 2.26, 2.10, 2.01, 1.95 and 1.94, with canonical 1.47 to 1.52 (Table 15).

For spelling, typo detection with matched controls clears in no model (Y19 passes). The mixed head spells 0.66 of 500 words correctly against 0.99 for the unmodified model (Y21 fails), and a character-only head reaches 0.94. Arithmetic is untested and listed as a limitation.

Thank you again for a careful review. The second model family, the added tasks, the trunk and head-input controls, the mixing ratios and the spelling test all answer questions you raised. Several of the results went against claims in the first version, and the revision drops or narrows each of those claims. If these corrections lead you to reconsider your scores, we would be grateful; if not, we understand. We are happy to discuss any of these results before the discussion period ends.

Thank you so much for the detailed revision and for running the additional experiments. However, the revised results no longer support the paper's two headline claims, and some of the framing has not been updated to match:

  1. "Learned late." Under matched controls, none of the three added runs is selectively acquired, and the original OLMo-2-7B word-order run also falls below the registered margin (final gap 0.04). In Amber-7B the controls rise before the task probe does. What remains is a general increase in transfer over training within OLMo-2, which is not specific to the task. This makes clear that “learned late” overstates what the evidence supports.

  2. "Broken at the head." Rank-16 trunk adapters (1.54 bpc) and a rank-64 map at the head's input (1.78 bpc) repair writing as well as or better than head retraining (1.94 at 2000 steps; 1.77 at convergence). The paper itself now states that where the failure originates remains open.

Given this, the title ("Tokenization Invariance Is Learned Late and Broken at the Head") no longer supports the paper's own findings and should be revised.

The findings that survive are the read/write asymmetry, which holds across all eight models, and the trunk's contribution to repair.

Overall, the revision improves the paper's soundness, and I am raising my soundness score to reflect this. Because the headline claims are no longer supported and the remaining positive results are narrow, I am keeping my overall rating. I appreciate the care and transparency reflected in the revision, and I hope the remaining issues can be addressed so the paper’s contributions are presented as clearly as possible. Thank you for your efforts throughout the discussion.

Reviewer_PWPCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work investigates whether language models retain task-relevant information when their inputs are represented using a different tokenization from the one used during pretraining. Using frozen linear probes across pretraining checkpoints, the authors find that word-order information becomes transferable to character-split inputs substantially later than it becomes decodable under canonical tokenization. They also show that the transfer is not necessarily construct dependent, with word-order information transferring more robustly than semantic-content and syntactic-acceptability information. They finally perform a series of language-modeling experiments and interventions that help them argue that the substantial degradation in character-split generation can be partly attributable to the output head, which can be retrained to recover much of the lost generation performance while keeping the pretrained trunk frozen. Overall I find the paper interesting and appreciate the extensive empirical analysis, the topic in general is conceptually useful given how foundational tokenization is to training and deploying language models. My main concern is whether the evidence fully supports the paper’s broader claims about the source of generation failures and the generality of its findings.

Critical strengths
  • Although the problem of tokenization invariance is not unique, the authors go beyond behavioral evaluations to examine whether task-relevant information remains accessible across canonical and non-canonical tokenizations. Using frozen linear probes, the matched random-direction controls and comparisons across pooling rules strengthens the analysis by helping distinguish task-specific transfer from generic representational stability.
  • The analysis they do is quite rigorous and they do so across training trajectories and model families. By analysing multiple checkpoints through pretraining and extending their analysis to several model sizes and families, the authors investigate whether word-order information transfers across tokenizations, but also how this transfer develops over training.
  • Their overall story seems quite connected, as they connect their probing results to language-modeling performance and interventions on the output head and trunk. In particular, the head-retraining experiment provides an interesting demonstration that character-split generation can be substantially improved without updating the pretrained trunk as they define it.
Critical weaknesses
  • I believe that the trunk-gap analysis does really isolate the contribution of the pretrained trunk. The real-trunk condition retrains a pretrained output head, but the random-trunk control randomly initializes both the trunk and head. So the 3.32 bpc gap that you report could partly reflect the benefit of starting from a pretrained head, rather than the contribution of the pretrained trunk alone. If you want to isolate the trunk’s contribution, you would actually compare a pretrained trunk with a randomly initialized head against a randomly initialized trunk with a randomly initialized head; this keeps the head-training procedure consistent across conditions.

  • The comparison between the head retraining and adaptation of the trunk doesn't seem to fully prove that the output head is the primary source of the generation failure. Although the two interventions have similar numbers of trainable parameters, they seem to differ in in where and how the model is adapted; head retraining changes the mapping from hidden states to output tokens, but LORA control adapts the trunk and keeps the head frozen. The trunk-only intervention also degrades canonical-language modeling, with bpc increasing from 1.31 to 6.73, while the head-retraining intervention preserves canonical performance (1.31 to 1.54). his difference makes it difficult to determine whether the weaker character-split performance reflects the limitations of trunk adaptation itself or the particular trade-off induced by this intervention.”

  • The authors show that retraining the output head significantly improves character-split generation, however it does not seem to reliably improve the model’s accuracy on prompted tasks with character-split inputs. To me, this raises the question of why the intervention improves generation without yielding comparable improvements in task performance, and whether additional interventions are needed to address both.

  • I really appreciate the extent of analysis that the authors have done in this work, however I am still curious about the generalization of their findings across model families that might have different pretraining trajectories. The OLMo-2-7B trajectory shows that robust word-order invariance emerges substantially later than canonical-tokenization performance; whereas the two Pythia trajectories do not reach the same invariance threshold within their observed training runs. Can we really ascertain that invariance is generally learned late during pretraining or whether its emergence depends on the model family and other training artifacts.

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer PWPC

Thank you for your positive and specific review.

You were right about the trunk gap and head initialization. In the 2x2 with matched initialization, Llama-3.1-8B's trunk gap is 2.07 bpc, down from 3.32 (updated Table 10), and it still rises with retention across eight models.

We no longer call the head the primary source (and uploaded a revised pdf). The paper claims only that it is one sufficient repair site, and the trunk is an equally effective one. For the LoRA trade-off, our new controls report character-split and canonical bpc together for every configuration (Figure 3, Table 12). A low-learning-rate LoRA at rank 16 repairs writing better than the head at 2000 steps (1.54 against 1.94) with canonical reading intact (1.49 against 1.54). The dense arm is close (1.86 at canonical 1.72), and a rank-64 map at the head's input ties a retrained head at convergence (1.78 against 1.77). The published LoRA failure at canonical 6.73 came from its training configuration, too bad :(

We also now realised that head retraining repairs writing but does not repair prompted reading. In OLMo-2-7B, where reading drops (0.92 to 0.77 on SST-2), a mixed-stream head and a mixed-stream trunk adapter on 872 SST-2 and 1066 Rotten Tomatoes sentences give 0.77 and 0.71 on SST-2 against the unmodified 0.77, so neither repairs reading (updated Table 13). After the registered run we extended this, post hoc, to OLMo-2-1B, 13B and Pythia-6.9b with rules fixed before each run (see Appendix C). No head and no mixed-stream adapter repairs reading in any of the four models and the adapters were checked over three seeds. A character-only adapter repairs it in OLMo-2-1B in two of three seeds (0.49 to 0.61 on average), at a cost to canonical writing (1.31 to about 2.2 bpc), and in no other model. With the trunk frozen, a head can change prompted accuracy only through its two label rows. Ten seeds of one head give 0.50 to 0.73, so single-run accuracies of retrained heads should be read with care.

We appreciate to put us on the path to test generalization across families! We added Amber-7B (13 checkpoints to 1.26T tokens) and an adjacent-swap order task, dated with a rule requiring the gain over matched controls to be selective, and Pythia is now described as censored. None of the three runs is acquired. Amber's word-order transfer rises, but its controls rise sooner, and adjacent swap shows no selective gain in either model. The paper now reports only a late rise of transfer, restricted to OLMo-2, and the seventy-fold number is removed.

Your points on head initialization and the LoRA trade-off shaped several of the new controls, and the revised paper makes narrower claims that are better supported! Thank you. If any of these corrections make you reconsider the rating, we would be grateful - if not, we also understand that this paper should be resubmitted.

Thank you for your response and for taking the time to address some of my concerns. I appreciate the additional experiments and the effort to revise the paper's claims in light of the new findings. I will maintain my original score.

Reviewer_dJw5OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents interesting findings on tokenization invariance in LLMs. The authors find that word-order information are obtained early in pretraining, while its representation becomes robust to character-split tokenization much later. The authors argue that the substantial degradation in character-level generation is primarily attributable to the output head.

Critical strengths

This paper presents interesting findings on tokenization invariance in LLMs. The authors find that word-order information are obtained early in pretraining, while its representation becomes robust to character-split tokenization much later.

Critical weaknesses

The paper suffers from insufficient clarity and coherence in its presentation.

The paper assumes substantial prior knowledge of tokenization, representation probing, and related evaluation methodologies. Several key concepts, experimental settings, and conclusions are introduced without sufficient background, motivation, or intuitive explanation, making the paper difficult to follow for readers who are not already familiar with this specific research area.

More importantly, the logical connections among the research questions, experimental designs, and conclusions are not clearly established. For example, the paper does not sufficiently explain why word-order, semantic-content, and acceptability probes are selected to characterize tokenization invariance, or how these measurements relate to the subsequent experiments on output-head adaptation. Similarly, the roles of different experimental controls, particularly the random-trunk baseline, are not clearly motivated in relation to the central research question.

Consequently, readers are required to reconstruct much of the paper's reasoning to understand what each experiment is intended to demonstrate and how the findings collectively support the main claims. This substantially increases the difficulty of evaluating the technical contributions.

Questions for authors

I am not very familiar with this specific research area, and I found it challenging to follow the paper's logic despite spending considerable effort trying to understand it. One question that particularly affects my assessment is the practical significance of the findings: To what extent could the proposed insights improve LLM performance or efficiency? In particular, what concrete benefits could these findings offer beyond the character-split setting?

4 comments
AuthorsOpenReview ↗

Response to Reviewer dJw5

Thank you for saying plainly that the paper was hard to follow. We think the fault is ours.

To make the paper clearer and its logic easier to follow, Section 2 now opens with one sentence followed through both tokenizations and says in plain words what a frozen probe, recentering, retention, controls and bits per character are. It explains why each task was chosen and why character splitting most directly threatens word order. Section 5 explains that the random trunk shows what a head can learn from character statistics alone, so the difference to the real trunk is what the trained network contributes. Table 1 maps each question to its experiment, its claim and the result that would refute it.

We tested practical significance, and the result is negative. A retrained head does not help prompted reading in any of the four models we tested (Appendix C), and on a spelling task the mixed head is correct on 0.66 of 500 words against 0.99 for the unmodified model (Y21 fails). The abstract now says that retrained heads spell worse than the unmodified model. Writing can be cheaply repaired at several places in the network, which suggests that tokenizer surgery need not retrain the whole model to recover writing, but we have not tested that. Beyond full splitting, re-cutting text at another tokenizer's boundaries costs almost nothing (Fig. 12), and probe transfer is graded in the fraction of tokens split.

Thank you very much for your thoughtful response and for taking the time to clarify the structure and main contributions of your paper. After reading the comments from the other reviewers, I realized that my difficulty in following the paper may have largely stemmed from my limited background knowledge in this particular research area. I appreciate the authors' efforts to address my concerns and improve the clarity of the manuscript. I also noticed the positive feedback from the other reviewers, which suggests that the work has considerable merit. I sincerely wish the authors the best with this research and hope to see your work receive the recognition it deserves at a top-tier conference in the near future.

AuthorsOpenReview ↗

Response to Reviewer dJw5

Thank you for your thoughtful feedback and for taking the time to reconsider our work. We are glad that our response has helped clarify the paper and address your concerns. In light of these clarifications, and your recognition of the merit of our work, we would be very grateful if you would consider whether your initial assessment of Weak Reject (2) might merit revision.

Thank you for your response. After carefully considering all the feedback and clarifications, I have decided to maintain my initial rating of Weak Reject (2), primarily because of the limited time available for further revisions before the camera-ready deadline. I would like to emphasize that my decision does not imply that I consider this research to be of limited value. Rather, I believe that promising research deserves sufficient time for discussion, refinement, and careful presentation so that its contributions can be communicated as clearly and convincingly as possible. I sincerely appreciate your efforts throughout the rebuttal process and wish you all the best with this work.

Reviewer_AygkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when representations become robust to character-level retokenization and where the resulting likelihood degradation originates. In OLMo-2-7B, canonical word-order decodability appears well before recentered probe transfer. Head-only adaptation substantially repairs character emission, while the authors argue that surviving invariance favors positional structure over semantic content.

Critical strengths
  1. The gap between learning a task and becoming robust to retokenization is interesting. The delayed improvement is also visible at fixed layers in the released results, so layer selection alone does not explain it.

  2. The paper goes beyond probing with useful interventions and reports its negative results openly. The distinction between character emission, canonical continuation and prompted accuracy helps establish what head retraining actually repairs.

Critical weaknesses
  1. Head-only repair shows that the frozen trunk contains useful features, but does not establish where the original failure resides. Appendix E and the code use a pretrained head with the real trunk and a random head with the random trunk, so the reported "trunk gap" changes both factors. The LoRA control also uses a different optimization problem with the same learning rate, and joint adaptation performs worse than head-only training.

  2. In the released Llama NLI result, canonical task accuracy is 72.9%, versus 95.5% for the random controls, yet selectivity is judged by their absolute transfer-accuracy difference. Task and control probes also use different reference data for recentering.

  3. I think the head-repair result is interesting, but I am not fully convinced by the claim that the failure is localized to the head. Successful head-only repair shows that the frozen trunk still contains useful information. It could also mean that the new head learns to compensate for changes in the representations. The comparison with trunk-only and joint adaptation matters here, but using the same learning rate for all three methods does not necessarily make that comparison fair.

  4. I am also unsure about the NLI selectivity result. The canonical accuracies in the released result are 72.9% for the task probe and 95.5% for the random controls. If these are the probes used in the comparison, the difference in transfer accuracy is hard to interpret because they start from quite different levels. The different reference data used for recentering adds another complication.

Questions;

  1. The abstract mentions a matched head-initialization experiment. How sensitive is the trunk gap to the initialization choice, and does it remain when both arms are trained to convergence? 2 Would trunk-only or joint adaptation become competitive with head-only repair if each method had its own learning-rate tuning? I would like to see character loss and canonical retention compared under similar compute budgets.
  2. Do the semantic-content results still hold when the control probes are matched to the task probe’s canonical accuracy and both use the same recentering procedure?
  3. How consistent is the model-size trend at matched token budgets? In particular, do probe retention and head-repair performance give the same ordering once the 13B result and the different pooling choices are included?
Questions for authors
  1. Does the trunk gap persist when both arms use the same head-initialization procedure and are trained to convergence?

  2. Can separately tuned learning rates make trunk-only or joint adaptation competitive with head-only repair? Please compare character loss and canonical retention under comparable compute budgets.

  3. Does the semantic-content conclusion survive matching control probes to canonical task accuracy and using identical calibration procedures?

  4. At matched token budgets and training stages, does larger model size consistently bring earlier invariance? In particular, does the size ordering agree between probe retention and head-repair performance once the 13B anomaly and pooling sensitivity are included?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Aygk

Thank you for reading our code and results so closely. We have carefully checked each of your points. We realise that there were some issues in our methodology and we have taken the opportunity to address them. We also realise that the paper was not in the best shape that it could have been and understand the rating.

You were right that the trunk gap confounds head initialization. We ran the 2x2 (real or random trunk, pretrained or fresh head) for all eight lineages with three random-trunk seeds, training each arm to convergence with its learning rate chosen on held-out data. The published rung reproduces in all eight (within 0.05 bpc). In Llama-3.1-8B, fresh and pretrained heads on the real trunk reach 1.79 and 1.77, and the trunk gap is 2.07 (fresh) and 2.09 (pretrained diagonal), against the published 3.32. Across eight lineages the gap rises with word-order retention (rho = 0.86, p = 0.005; informed, because retention was already known), and all registered secondary tests hold (we have updated Section 5, Table 10).

We also agree that the results do not show where the failure resides. The paper now says "repairable at the head" and makes no claim about origin anymore. For the LoRA learning rate and joint arm, we swept rank and learning rate, selecting on held-out chunks, and added a dense top-two-block arm, a joint arm with separate learning rates, a mixed-stream arm and 8000-step runs. Rank 16 at 10−510^{-5} reaches 1.54 bpc at canonical 1.49 at 2000 steps, against 1.94 and 1.54 for head-only. A second seed gives 1.56 and 1.47. The dense arm reaches 1.86 at canonical 1.72, and the joint arm reaches 1.57, better than head-only by more than our margin. The trunk is an equally effective repair site and adds to head repair, so we removed the comparative sentence. The published 6.73 canonical was an artifact of the training configuration. The advantage depends on stopping early, as the 8000-step run overfits (1.82 at canonical 1.75). Compute per arm is in an updated Table 12.

Your point on content selectivity was correct, and we have investigated that. The 0.10 margin was unreachable in several cells. Controls are now matched to the task's canonical accuracy within 0.02, task and controls share one recentering reference, the margin scales with headroom, and absence needs the interval to exclude it. MultiNLI clears under both pooling rules in GPT-2 and Llama-3.1-8B, Rotten Tomatoes sentiment only in Llama-3.1-8B, and word order under both in neither OLMo-2-7B nor Llama-3.1-8B. The claims that word order survives selectively and that positional structure survives "rather than content" are withdrawn. Matched controls reach the trim cap for word order and are then scored on a fifth of the rows, so the comparison favors the controls. We report fixed-trim gaps alongside (in updated Tables 7 and 8). We apologise for this. We should not expect a reviewer to find failures in our methodology, therefore we are grateful for your effort in pointing them out to us.

For Q4 (size ordering, 13B, pooling), 13B's final reading was set by our selection rule: Eight layers tie exactly on canonical accuracy, the rule took the lowest (0.60), and the others read 0.67 to 0.90. With the median over tied layers, row 12 strengthens to ρ=0.95\rho = 0.95 (last-token 0.93). The dip at 13B's 630B checkpoint is not a tie, but that checkpoint reads canonical text at 4.50 bpc against 1.33 at the checkpoints on either side of it.

If any of these corrections make you reconsider the rating, we would be grateful - if not, we also understand that this paper should be resubmitted.

Thank you for the additional experiments. I appreciate the corrections, but I still do not think the paper is in sufficiently good shape for acceptance. The updated results change several central claims, and I think these changes need to be worked through more fully. In my view, a thorough revision and resubmission would be more appropriate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.