- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper examines whether differences in target-token composition can distort layer analyses and produce misleading evidence of depth specialization. It introduces a cross-corpus contrast between early- and late-layer ablations, breaks it down by token type, and reevaluates it using shared target mixtures. In its comparison, whitespace targets drive the positive pooled contrast: removing them reverses its sign in all three fully decomposed models, while matching to a reference mixture reverses it in two. The paper also presents a tokenizer-only risk screen and a reporting protocol.
Critical strengths
S1. The paper identifies a relevant measurement confound. It clearly explains why replication across models does not establish task specialization when the evaluated corpora share the same differences in token composition.
S2. The central empirical finding undergoes extensive and rigorous scrutiny. The evaluation spans several model families and corpus pairs, with checks covering position matching, token-classification rules, continuous readouts, and perturbation strengths.
S3. The proposed reporting protocol is practical. Recording target types, reporting conditional effects, and comparing shared-mixture estimates can improve interpretability analyses using existing prediction data.
Critical weaknesses
W1. The main finding relies on a restricted setting. The analysis uses next-token logits to score tokens three positions ahead, while late-layer mean-blend causes the model to predict whitespace at a high proportion of GSM8K positions. Although other horizons and operators are examined, their patterns differ. Stronger evidence connecting this diagnosis to standard next-token prediction and task performance would clarify its practical relevance.
W2. Correctness matching requires cautious interpretation. Reweighting by unperturbed correctness (Section 5) may remove meaningful capability differences, so attenuation after matching alone does not rule out task-related depth specialization.
Questions for authors
See weakness
2 comments
Thanks to the author's effort in addressing my concerns. I don't have any further questions now. And I will maintain my score.
Reply to Reviewer Av4W
Thank you for the careful reading and positive assessment. We address your two concerns below, using the new measurements summarized in the general response.
W1: standard next-token prediction and task performance.
We extended the mean-blend analysis to H=1–8 and compared both operators at H=1 on GSM8K/Wikitext. The mean-blend sweep uses five model instances, 300 sequences per corpus and 2,000 sequence-bootstrap draws. The H=1 mild-resample analysis uses 250 sequences per corpus.
The audit therefore remains informative at ordinary next-token prediction, although the relevant composition factor and outcome depend on the intervention. Persistence after this adjustment does not rule out other composition effects.
We did not measure generation-based task accuracy. Our evidence concerns token-level contrasts under the tested interventions, not a demonstrated bias in end-task performance. Establishing that connection requires a separate evaluation.
W2: matching on unperturbed correctness.
We agree that matching on unperturbed correctness can remove meaningful capability-related variation. Attenuation after matching alone therefore does not rule out task-related depth specialization. We interpret the matched estimates as sensitivity checks and retain the unadjusted results.
To examine the mechanism beyond attenuation, we inspect the initially correct content-word predictions in the H=1 mild-resample series. For each corpus, the removal gap is the fraction of these predictions lost under the early band minus the fraction lost under the late band. The product below multiplies the mean of the two removal gaps by the difference in the control-correct shares of content targets. These diagnostics use the paper's sequence-averaged convention.
The baseline accuracy difference is observed directly: the unablated model predicts a larger share of GSM8K content targets correctly. The removal gaps are similar across corpora in four instances, with a larger difference in Llama-3.2. The product is comparable in magnitude to the content-word CAI, but it is a descriptive calculation, not an exact decomposition or proof that the baseline difference is unrelated to capability. Llama-3.2 also retains a smaller positive pooled contrast after matching, as reported under W1.
Two further checks help delimit the interpretation:
Our conclusion is that the pooled contrast is sensitive to which targets are evaluated and how predictable they are before intervention. The audit makes that sensitivity explicit. Neither attenuation nor persistence after matching, on its own, establishes whether the model has task-dependent depth specialization.