acceptodds
Under review as a conference paper at ICLR 2027

Persistent Depth Ordering amid Shifting Block-Bypass Responses in Language Model Pretraining

Abstract

Layer interventions are widely used to probe the internal organization of language models, yet most analyses examine a single training checkpoint even though model representations and computations evolve throughout pretraining. This leaves open which depth-dependent intervention responses reflect persistent organization and which are transient consequences of training. We study this question using single-block identity bypass on fixed teacher-forced contexts across five released trajectories and 11 model–domain combinations. We find that block-bypass responses retain recognizable depth ordering while their magnitudes redistribute: nearby checkpoints preserve stronger rank correspondence than distant ones, and large changes concentrate at positions that recur across text samples and transfer across evaluation domains. Controlled experiments further show that changes in the natural bypass effect cannot be reduced to a single downstream sensitivity: in replicated Pythia runs, local missing-update magnitude grows while the pooled matched downstream response decreases, whereas OLMo-2 7B exhibits a different balance. These matched responses also depend on perturbation strength and direction, without identifying targeted compensation. Together, our results show that longitudinal layer sensitivity is structured but not static, and that single-checkpoint intervention responses should be interpreted in the context of how the underlying perturbation pathway evolves during training.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, pretraining dynamics, layer interventions, block bypass, longitudinal analysis, depth ordering, language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kf2ZOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how the importance of a single layer in a transformer is for next-token predictions changes over training checkpoints. They test most layers across various small open-source models at 3-5 checkpoints in the training, measuring the difference in prediction scores across general/math/code datasets. They find that the rankings of most to least important layers is strongly correlated between adjacent checkpoints and correlated between the earliest and latest checkpoints. Furthermore, most (65%) of the change occurs in 20% of the blocks, and patterns transfer across domains. Finally, they find that skipping a layer makes a bigger difference later in training partly because that layer performs a larger update to the residual stream, not necessarily because the model is more sensitive to that layer.

Critical strengths

None

Critical weaknesses
  • The significance of the results is under-motivated. Nothing about the experiments reveals what the blocks compute or why the block rankings change. Furthermore, the reason they are experimenting on bypassing blocks in the first place is not clear; there is no further interpretation of the revealed block ordering in terms of improving model interpretability or capabilities. They do not try to characterize the blocks or explain the reasoning behind their effect on the outputs.
  • Many parts of the paper, including figures, seem AI-generated with little human iteration. For example, parts of the abstract are cryptic and difficult to read ("single-block identity bypass on fixed teacher-forced contexts", "longitudinal layer sensitivity is structured but not static"). Furthermore, Figure 1, which should typically summarize the main result or contribution of the paper in a visually appealing way, is completely unintelligible to me: it is not clear from the figure or the caption why the information is being organized in these blocks with arrows between them, what "follow the checkpoint x depth response field" means, what 14k/72k/143k refer to, etc. These presentation issues, which continue throughout the rest of the paper, make it very difficult for a human to understand and assess the contribution of the paper. Therefore, the paper's current presentation does not meet the threshold for ICLR and should receive a clear rejection.
Questions for authors

Can the authors please explain, either from their existing results or from additional experiments, what can be said about how the model performs computation or develops capabilities? Is there any interpretability insight we can gather from these results?

2 comments
AuthorsOpenReview ↗

Response to Reviewer kf2Z

Thank you for the direct and detailed feedback. We agree with both concerns and have revised the paper accordingly.

Presentation. We agree that the submitted version made the contribution hard to assess. The phrases you quoted were unclear, and Figure 1 did not communicate its content. We rewrote the main text. The abstract and introduction now state the question and findings in plain language, every term is defined at first use, and Figure 1 is a new figure with a schematic and three data panels in which training progress is labeled explicitly. We would be grateful if you could look at the revised version.

Significance: why bypass blocks, and what we learn. The motivation is practical. Layer importance from skipping layers is widely used to decide which layers to prune or skip, and it is almost always measured on a single checkpoint. We ask whether such conclusions survive continued training. The revision makes this concrete (Section 4.6): in SmolLM2, removing the two blocks that an early checkpoint ranked least important hurt the final model far more than removing the two selected at the final checkpoint (perplexity 25.3 vs. 15.6), while in OLMo-2 1B the early choice was better. Rankings from one checkpoint are thus a fair guide to coarse depth organization, but they should be re-evaluated before specific layers are removed.

Q1: Interpretability insights. We do not claim to identify circuits, but the results bear on how ablation evidence should be read:

  1. The coarse depth ordering of block sensitivity persists through pretraining, while a few mostly shallow blocks change substantially, and the same blocks change on general, math and code text (Sections 4.1 and 4.3). Where we compared sublayers, these changes track the MLP more closely than attention (Appendix H).
  2. In exploratory multi-seed comparisons, part of a block ranking is specific to the training run, so a ranking from one run need not transfer fully to another run of the same model (Section 4.2).
  3. A larger ablation effect need not mean greater reliance on a block. In Pythia-160M the block's update grows while the response to an update of matched size falls, and other models show other patterns (Section 4.5). Studies that compare ablation effects across checkpoints should therefore report update size alongside the effect.

We also tested one simple explanation of persistence and found it unsupported in Pythia-160M, which the revision reports explicitly.

Thank you again for pushing us to make the contribution clear.

Thank you for the thoughtful response and revisions. I appreciate the efforts to improve the presentation and clarify the practical motivation, and will raise my score from 1 to 2. However, my main concern remains: the results provide limited insight into what the blocks compute or how capabilities develop, leaving the broader interpretability significance insufficiently established.

Reviewer_wqGCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper uses an identity bypass to replace some block of one model and evaluates the output perturbation. Using this mechanism, it can measure the importance of one block. Then it adopts this technique to open models, like Pythia and Olmo. It finds that the importance ranking of different depth blocks is kept while the scale can vary. Moreover, it finds that the rankings are shared across different domain data. It also explores the difference between the beginning and end of training to analyze the impact of identity bypass, and find the scale of block update can greatly affect the output changes.

Critical strengths
  1. The technique is interesting and simple to adopt widely in practice.
  2. The observations are interesting, such as the depth ordering of importance being kept across different datasets.
Critical weaknesses
  1. Lack of mechanism explanation. The current content is primarily an observational ensemble, while a further step would be an explanation of the persistent depth ordering.
  2. Poor writing. It is hard to follow the roadmap of this work, and Figure 1 is clearly AI-generated and out of logic.
  3. The validation is only performed on Pythia and Olmo. Some recent open models like Instella, SmolLM3, MobileLLM-R1, OLMoE, and Puro might be included for testing to cover a larger set and different types of models.
Questions for authors
  1. Can you provide some further steps analysis on explaining your core claims, like persistent depth ordering?
  2. Can you add an ablation study on more models, like some listed in Weaknesses?
2 comments
AuthorsOpenReview ↗

Response to Reviewer wqGC

Thank you for finding the technique simple and the persistence across datasets interesting. We have revised the paper substantially in response to your three points.

W2: Writing and Figure 1. We agree, and this was the most important problem to fix. The original Figure 1 tried to summarize the whole pipeline and was not readable. We replaced it with a schematic of the intervention and three panels drawn directly from the data, each making one point. We also rewrote the main text: the paper now follows one question, each subsection states its result first, a small set of terms is defined once, and detailed statistics are in tables and the appendix.

W1 / Q1: Explaining persistent depth ordering. We took this question seriously and tested a concrete explanation: that rankings persist because the ranking of update sizes, i.e. how much each block writes to the residual stream, is stable. In Pythia-160M this is not supported. The update-size ranking is uncorrelated between the first and last checkpoint (−0.16), while the bypass ranking is preserved (0.85), and update size does not improve held-out prediction of later effects (Section 4.5). We also found that changes concentrate in a few mostly shallow blocks (Section 4.3), and that the MLP sublayer's pattern of change tracks the whole-block pattern more closely than attention's (0.85–0.86 vs. 0.44–0.77; Appendix H). These results narrow the question but do not answer it, and the revision states openly that the mechanism of persistence remains open. We believe that reporting a natural hypothesis that does not hold is useful to readers.

W3 / Q2: More models. We realize the model coverage was not visible enough. The submission already included SmolLM2 1.7B on general, math and code text, and its pattern differs from OLMo-2 7B: large increases at a few blocks rather than decreases near the front. The introduction and Table 1 now state the coverage explicitly. Within the discussion period we prioritized replication across random seeds and added five Pythia-410M runs (Sections 4.2 and 4.5), because this tests whether our findings depend on a single training run. We agree that broader model coverage would strengthen the conclusions. Several of the models you suggest, including OLMoE and SmolLM3, release intermediate checkpoints, which makes them natural candidates for extending this analysis.

Thank you again; we believe the revision is much clearer as a result of your comments.

Thank you for the thoughtful response and the additional analyses. However, my main concerns remain. The underlying mechanism behind persistent depth ordering remains unexplained. I will maintain my original score of 2.

Reviewer_VHjxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses interventions on specific layers, which are commonly used in the analysis of language models. While major studies examine only a single checkpoint and don't address the changes during model training, this study examines checkpoints from publicly released training trajectories in order to investigate which characteristics change and which are preserved during training. The study adopts a fallback of each layer to the identity mapping as the intervention method, and uses the top-1 disagreement of the output as the observed measure. Each layer within a training trajectory is characterized by its index. As a result of this investigation, they observe that the ordering of highly influential layers changes little along the training trajectory, and that changes occur disproportionately in a subset of layers. In addition, in the controlled experiments on Pythia and OLMo-2, they point out that the local magnitude of the update written by each layer and the downstream response to a perturbation matched in magnitude to that update show different changes in these models.

Critical strengths
  • This study deals with training trajectories and can be regarded as discussing the robustness of single-checkpoint evaluation from a particular viewpoint. Within the scope of the experiments, when viewed from the perspective of output changes over time, the ranking of influence along the depth is relatively preserved, and the positions whose influence changes during training are similar across domains. This suggests that the measure reflects the structure of the model rather than the type of text/domain. In terms of providing a qualitative justification for the measure examined here, this appears to be a useful result.
  • Experiments are conducted on multiple model series, providing information on what is reproduced and what differs across models of entirely different origins. Through the discussion of the local missing update and the matched response, the paper clarifies that the underlying mechanisms can differ even for the changes appeared on the same downstream measure.
Critical weaknesses
  • The baseline for changes in the correlation coefficients is unclear. For example, even for the same checkpoint, changing the data changes the output tendencies. Such non-essential changes outside the aspect of interest need to be discounted so that only substantive changes are considered, but this is not clearly shown within the scope of the experiments.
  • The architecture/hyperparameter of the models examined here differ substantially, and these differences introduce many confounding factors. Therefore, it is challenging to discuss how general the differences between models observed in these experiments are. In particular, the Pythia 160M configuration is very small for a modern language model, and I think the authors should discuss the possibility that the model itself saturates before features sufficient to enable various analyses emerge along the training trajectory.
Questions for authors
  • Are the Pythia 160M "Runs" artifacts derived from PolyPythias? If so, please cite the literature properly.
1 comment
AuthorsOpenReview ↗

Response to Reviewer VHjx

Thank you for the constructive review, and for recognizing the value of comparing model families and of separating update size from the downstream response. Your comments led to two of the main additions in the revision.

W1: Baseline for changes in correlation. We agree that a change in correlation is only meaningful relative to what happens without training, and we added two baselines.

  • Same checkpoint, different text. We compute each ranking on a different half of the evaluation text. At the same checkpoint the halves agree at 0.885; between the first and last checkpoints, at 0.629 (difference 0.256, 95% CI 0.203–0.276; positive in all 11 settings). Figure 1c shows the comparison at every checkpoint distance (Section 4.1, Table 13).
  • Same step, different seeds. In five Pythia-410M runs, two halves of the text agree within a run at 0.85–0.92, clearly above the agreement between runs (0.77 at 14k, 0.51 at 143k). The six Pythia-160M runs show the same pattern (Section 4.2, Table 15).

The changes we report therefore exceed text-sampling variation, and part of each ranking is specific to the training run. We label the seed analyses exploratory, since they were added during the discussion period.

W2: Confounds across families, and the size of Pythia-160M. We agree that the families differ in architecture, data and schedule at the same time. The revision compares checkpoints only within a run and states that we do not attribute differences between families to any single factor (Section 1, Scope; Section 5). We should also have made clearer that the broad analysis covers models from 1B to 13B parameters; only the seed-replicated experiments used 160M. Following your suggestion, we repeated them on five PolyPythias-410M runs. Persistence and rising sensitivity replicate: sensitivity rises in all five runs. The response to an update of matched size, however, rises at 410M while it falls at 160M, so we narrowed that claim to 160M and report 410M as a counterexample (Section 4.5). Because the two sizes also differ in depth, numerical precision and training run, we do not attribute the difference to scale, and we do not claim that it reflects saturation.

Q1: PolyPythias. Yes, the Pythia-160M runs come from PolyPythias. We apologize for the omission; the revision cites van der Wal et al. (2025) where the runs are introduced (Section 3.2) and throughout.

We hope these changes address your concerns, and we would be glad to answer further questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response: summary of revisions

We thank all three reviewers for their careful and candid reviews. We agree with the two concerns raised most often: the submission was hard to read, and it did not make clear why longitudinal block sensitivity matters. We have uploaded a revised PDF. Below we summarize what changed; a follow-up comment maps every claim of the submission to its place in the revision.

  1. Rewritten main text. We rewrote the main text for readability while keeping the experiments and claims. The paper now follows one question: does layer importance measured on one checkpoint still hold as pretraining continues? Each subsection states its answer first, a small set of terms is defined once and used consistently, and detailed statistics are in tables and the appendix. Figure 1 is new: a schematic of the intervention and three panels drawn directly from the data. Because the text changed throughout, the pdfdiff is large; the claim map in the next comment shows that no result was removed.

  2. A text-sampling baseline (VHjx). We compute each ranking on a different half of the evaluation text. At the same checkpoint, the halves agree at 0.885; between the first and last checkpoints, at 0.629 (difference 0.256, 95% CI 0.203–0.276; positive in all 11 settings). Training therefore changes rankings by more than text sampling does (Section 4.1, Figure 1c, Table 13).

  3. Seed variation and a larger model (VHjx, wqGC). We added five Pythia-410M runs from PolyPythias. They reproduce the persistence of rankings and the rise in sensitivity. Comparing runs shows that rankings combine a part shared across runs with a run-specific part (Section 4.2, Tables 14–15). These analyses were done during the discussion period and are labeled exploratory.

  4. Narrowed and tested claims (all reviewers). At 410M, the response to an update of matched size rises, whereas it falls at 160M. We therefore state the 160M result as specific to that setting and report 410M as a counterexample (Section 4.5). We also tested a simple explanation of persistence, that stable update-size rankings produce stable block rankings; in Pythia-160M it is not supported, and we state that the mechanism remains open.

  5. A practical consequence (kf2Z). In SmolLM2, removing the two blocks that an early checkpoint ranked least important hurt the final model far more than removing the two selected at the final checkpoint (perplexity 25.3 vs. 15.6); in OLMo-2 1B the early choice was better (Section 4.6, Table 17). Rankings from one checkpoint are a fair guide to coarse depth organization, but specific layer choices should be re-evaluated on the target checkpoint and run.

We also added the missing PolyPythias citation, a SmolLM2 training-stage analysis, and checks for near-tied predictions and numerical precision (Appendix H). All new analyses use existing archives or released checkpoints.

AuthorsOpenReview ↗

Claim map: where each claim of the submission is in the revision

No experimental result of the submission was removed. The old workflow figure was replaced by the new Figure 1, and the old Figure 2 moved to Appendix B (Figure 6). Below, each claim of the submission is listed with its status in the revision.

  1. Block rankings persist across training while magnitudes change; nearby checkpoints agree more than distant ones. Retained, and strengthened by a text-sampling baseline and a 410M replication. (Sec. 4.1–4.2, Fig. 1c, Tables 13–15)

  2. Large changes concentrate at positions that recur on held-out text and transfer across text domains. Retained. We added that transfer requires substantial change (late OLMo-2 7B interval: 0.28) and a SmolLM2 stage split (65% of change within one stage). (Sec. 4.3, Figs. 2–3, Table 16)

  3. Continuous outputs show the same training effect and depth ordering. Retained. We added that much of the top-1 increase occurs at near-tied predictions, while continuous measures rise clearly. (Sec. 4.4, Fig. 4, App. H)

  4. In Pythia, local update size grows while the pooled matched response decreases; OLMo-2 7B shows a different balance. Narrowed to Pythia-160M; Pythia-410M is reported as a counterexample (the matched response rises). (Sec. 4.5, Figs. 1d and 5)

  5. Matched responses depend on perturbation strength and direction, without evidence of targeted compensation. Retained. (Sec. 4.5, App. E–F)

Added in the revision:

  • Rankings combine run-shared and run-specific structure (exploratory). (Sec. 4.2, Tables 14–15)
  • Stable update-size rankings do not explain persistence in Pythia-160M (exploratory). (Sec. 4.5)
  • Reusing an early ranking for layer removal (exploratory). (Sec. 4.6, Table 17)
  • Attention vs. MLP sublayers; numerical precision. (App. H)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.