- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
When a subword language model is given character-split input, its performance collapses, which is usually taken to mean the model is tied to its tokenizer. This paper argues instead that the failure lies mainly in the output head, while the rest of the model still understands the input. Probing fifteen pretraining checkpoints of OLMo-2-7B with frozen linear probes and matched random-direction control probes, it finds that the model learns a word-order task by 9B tokens but only becomes robust to character splitting around 630B tokens, and that this robustness holds for word order but not semantic content. Retraining only the output head of Llama-3.1-8B over a frozen trunk restores generation performance to near-canonical levels, and the trunk’s contribution to that repair correlates with probe-measured invariance across eight models.
Critical strengths
One of the major strengths of the paper is its transparency. The paper preregistered 20 quantitative predictions and openly reports all outcomes (8 passed, 12 failed), giving a clear picture of the findings. The read/write asymmetry is also a useful finding. Models read character-split text at modest cost but cannot produce it, and a head retrained over the real trunk restores the performance to near-canonical while one over a random trunk cannot. It rests on direct intervention rather than probe interpretation. It gives tokenizer-transfer and byte-level conversion work a concrete target, and it is the main reason I see real value in the paper despite my concerns about the probing claims.
Critical weaknesses
The main claims are much broader than the experiments behind them. The 70× delay comes from one training run of one model (OLMo-2-7B) on one task (detecting shuffled English sentences). I don’t see how this supports a general statement about when models learn tokenization invariance. Would the same delay appear with a different positional task or in another model family with checkpoints past 500B tokens? The Pythia comparison doesn’t help here: both Pythia runs stop at 300B tokens, before OLMo-2-7B itself crossed the threshold (428B), so we can’t tell whether Pythia never gets there or just hasn't yet. Likewise, "content does not survive" is tested with a single NLI task, and "syntax" with BLiMP alone. These are claims about three datasets, not about constructs.
I also have some concerns about the trunk adaptation controls. To test whether adaptation is localized specifically to the head, the authors trained LoRA adapters on Llama-3.1-8B's trunk while keeping the head frozen. The trunk arm degraded canonical text reading from 1.31 to 6.73 bpc (Table 2, Row 20; Table 7). Because this control severely damaged standard reading capabilities, it leaves open whether trunk adaptation is inherently incapable of character generation or if the LoRA training setup caused catastrophic interference. Without a working control, the claim that the failure is localized to the head rests on the head experiment alone.
Questions for authors
-
Does the late-invariance finding hold for a second positional task, or for another model family with checkpoints beyond 500B tokens? Forward-pass mean translation patching removes at most 27% of the collapse at the best single layer. Why can't the unretrained output head process this shifted trunk geometry without explicit parameter updates? (Figure 8)
-
Trunk LoRA adaptation degraded canonical reading from 1.31 to 6.73 bpc. Did you test lower ranks (e.g., r=16) to confirm whether this degradation is an artifact of the high rank (r=192, 503M params) rather than a trunk limitation? (Table 7)
-
A 100% character head drops SST-2 accuracy (0.70 to 0.55) on character-split prompts, while a 50/50 mix preserves accuracy but leaves bpc elevated (2.80 vs. 1.63 baseline). Did you evaluate asymmetric mixing ratios (e.g., 90/10) to improve this trade-off?
-
Since trunk invariance holds for word-order shuffling but fails for semantic content (NLI) and syntax (BLiMP), does this invariance generalize to fine-grained character-level or symbolic tasks (e.g., spelling or arithmetic)?
Happy to discuss these questions during the rebuttal period, and I am open to reconsidering my scores if my concerns get addressed.
2 comments
Thank you so much for the detailed revision and for running the additional experiments. However, the revised results no longer support the paper's two headline claims, and some of the framing has not been updated to match:
-
"Learned late." Under matched controls, none of the three added runs is selectively acquired, and the original OLMo-2-7B word-order run also falls below the registered margin (final gap 0.04). In Amber-7B the controls rise before the task probe does. What remains is a general increase in transfer over training within OLMo-2, which is not specific to the task. This makes clear that “learned late” overstates what the evidence supports.
-
"Broken at the head." Rank-16 trunk adapters (1.54 bpc) and a rank-64 map at the head's input (1.78 bpc) repair writing as well as or better than head retraining (1.94 at 2000 steps; 1.77 at convergence). The paper itself now states that where the failure originates remains open.
Given this, the title ("Tokenization Invariance Is Learned Late and Broken at the Head") no longer supports the paper's own findings and should be revised.
The findings that survive are the read/write asymmetry, which holds across all eight models, and the trunk's contribution to repair.
Overall, the revision improves the paper's soundness, and I am raising my soundness score to reflect this. Because the headline claims are no longer supported and the remaining positive results are narrow, I am keeping my overall rating. I appreciate the care and transparency reflected in the revision, and I hope the remaining issues can be addressed so the paper’s contributions are presented as clearly as possible. Thank you for your efforts throughout the discussion.
Response to Reviewer B36Q
Thank you. We have carefully checked each of your points. We realise that there were some issues in our methodology and we have taken the opportunity to address them. We also realise that the paper was not in the best shape that it could have been and understand the rating. We have updated the PDF, but of course we do not expect a second review. We've summarised the changes below:
To address one run, one task and one family, we added Amber-7B and an adjacent-swap task with a registered dating rule requiring selectivity. None of the three added runs is acquired, so "seventy times" is removed and the late-transfer statement is restricted to OLMo-2. The only ratio available is the stored OLMo-2-7B one, 47.6, with fixed-trim controls. Pythia is described as censored.
We agree that three datasets do not establish constructs, and the paper now names tasks. We added Rotten Tomatoes sentiment, CoLA acceptability and typo detection. Sentiment clears under both pooling rules only in Llama-3.1-8B, CoLA and BLiMP clear under both in no model, and typo detection clears in no cell (updated Table 3).
For the LoRA that damaged canonical reading, and for lower ranks, we ran ranks 16, 64 and 192 at three learning rates, with a dense arm and a mixed-stream arm. A working control exists (W passes: 3 of 14 configurations). Rank 16 at 10−5 reaches 1.54 bpc at canonical 1.49, better than head-only (1.94), so the head has no advantage in this comparison and we remove the comparative sentence.
The unretrained head cannot use the shifted geometry (Fig. 8) because the mean shift fixes only the first moment. We tested this at the head input directly. A mean shift reaches 16.19 bpc, while a full affine map and a rank-64 map reach 1.78, against 1.77 for the retrained head (Y6 fails). A map at the head input is enough to repair writing.
For mixing ratios, character shares from 0.1 to 0.9 give character-split bpc of 2.26, 2.10, 2.01, 1.95 and 1.94, with canonical 1.47 to 1.52 (Table 15).
For spelling, typo detection with matched controls clears in no model (Y19 passes). The mixed head spells 0.66 of 500 words correctly against 0.99 for the unmodified model (Y21 fails), and a character-only head reaches 0.94. Arithmetic is untested and listed as a limitation.
Thank you again for a careful review. The second model family, the added tasks, the trunk and head-input controls, the mixing ratios and the spelling test all answer questions you raised. Several of the results went against claims in the first version, and the revision drops or narrows each of those claims. If these corrections lead you to reconsider your scores, we would be grateful; if not, we understand. We are happy to discuss any of these results before the discussion period ends.