acceptodds
Under review as a conference paper at ICLR 2027

Final-Layer Jumps in Transformer-LMs: Causes, a Remedy and Interpretability Effects

Abstract

Transformer language models (LMs) exhibit a final-layer jump: hidden states change gradually across the middle layers but shift abruptly at the final layer, a pattern widely observed in existing Transformer-LMs. The jump matters for Vocabulary Projection Methods such as the Logit Lens, which read out intermediate hidden states with the final LM head. Since the LM head is trained to decode the hidden state after the jump, a large final-layer change can reduce compatibility between intermediate hidden states and the LM head, degrading readouts at all intermediate layers. Through architectural ablations, we localize the jump to the final feed-forward network and show that it can be relocated: a linear buffer layer inserted before the LM head absorbs the jump, yielding a jump-relocated counterpart. Comparing models with and without the jump, we find that intermediate-layer representations remain highly similar while Logit Lens readouts change substantially, showing that the final-layer jump alters the results of Vocabulary Projection Methods and should therefore be taken into account.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Transformer-based Language Models, Hidden States, Mechanistic Interpretability, Architecture

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Ymt3OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explains that the performance of Vocabulary Projection Methods (VPM) is degraded due to the 'final-layer jump' symptom that occurs within Transformer layers. To improve the interpretability of vocabulary-space readouts from hidden layers, the authors propose adding an additional linear layer before LM head at every transformer layer. The paper empirically shows that the extra layer effectively mitigates the final-layer jump, resulting in making the hidden state better aligned to the output of the final layer. Several ablation studies further support the effectiveness of the proposed architectural change.

Critical strengths
  1. This study finds out the causal relationship between the final-layer jump symptom and Transformer architecture. Through heavy empirical study, the authors argue that the symptom is generally found in many Transformer-based LLMs.

  2. Thorough and extensive ablation studies provide useful insights. Appendix also includes a wide range of experimental results.

  3. The proposed solution is simple and can be easily applied to any kinds of Transformer models.

Critical weaknesses

Although I see useful empirical results and interesting explanations of the observed results, I also see several non-negligible limitations as follows.

  1. [major] First of all, although it is interesting that the final-layer jumps are the main reason for the performance drop in VPM, this cannot be considered as a significantly impactful finding. In Section 3, the authors comprehensively investigate what makes the final-layer jumps occur and where they are often observed (final linear transformation layers at each block). This symptom becomes problematic in a very specific situation where the hidden state representation is used to extract the output tokens (e.g., LogitLens). Therefore, I consider the contribution is not sufficiently strong to provide valuable insights to the ICLR readership.

  2. [major] Inserting a buffer layer into Transformer blocks may relocate the layer where the final-layer jump occurs, however, it is a simple (as the authors stated in the paper) trick rather than a meaningful architectural advancement. In Table 18, the Commonsense suite performance is consistently degraded though the drop is marginal. Why would inserting a single buffer layer right before LM head be the optimal solution? Why not adding more layers with smaller number of weights (e.g., LoRA-like bottleneck layers)? This simple trick should be more rigorously substantiated.

  3. [major] While the proposed buffer layer relocates the final-layer jump, the underlying mechanism responsible for this effect remains unclear. The generality of this effect is still not theoretically grounded. The empirical study relies only on a limited set of benchmarks and a few models. Thus, the proposed approach should be viewed as an empirical observation rather than remedy to a certain problem.

  4. [major] In Section 3.3, the authors state that no layers other than the reported ones have ζl\zeta_l larger than 3.0. However, Figure 2(d) shows ζ15=7.56\zeta_{15} = 7.56 for the ReLU variant, which means the jump occurs in both of the last two layers for this model. The authors should correct this statement and explain why the ReLU variant also exhibits a jump.

  5. [minor] The font size in most figures should be increased. It is hard to read the text, especially Figure 6.

  6. [minor] The text in Section 3.3 lists the jump scores 9.26, 6.13, 16.00, and 5.49 without indicating which model each value belongs to. I recommend explicitly mapping those scores with corresponding models (methods) in the main text so that the paper becomes self-contained.

Questions for authors

My questions and concerns are all summarized in the above weakness section. Please carefully address them.

Reviewer_aUxYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates final-layer jumps in Transformer LLMs and shows through architectural ablations that they primarily occur in the final FFN rather than the attention module, with their location strongly influenced by the FFN's two-linear parameterization.

Motivated by this finding, the authors introduce a linear buffer before the LM head to relocate the jump outside the Transformer layers. Using the buffer and LM head as a readout increases the Top-K overlap with final predictions and changes previous Logit Lens interpretations of multilingual models.

Critical strengths

The systematic ablation studies identifying the location of final-layer jumps are the strongest contribution. In particular, it is interesting that jumps are localized to the final FFN and that single-linear and two-linear FFNs exhibit different jump locations despite having the same function class. The finding that an additional linear buffer can absorb the jump is also intriguing.

Critical weaknesses

The necessity and practical value of the proposed buffer layer are not sufficiently justified. The original final Transformer block may already serve as a nonlinear buffer. Could similar results be obtained simply by applying the original final block followed by the LM head to intermediate hidden states? The increased Top-K overlap in Section 5 may be a natural consequence of incorporating additional output-side transformations into the readout. Similarly, if the late-layer jump in the original model partly performs translation into the target language, relocating this transformation to the buffer could naturally explain the earlier emergence of Chinese tokens. While these results demonstrate the dependence of interpretation on the readout, the unique benefit of introducing a separate buffer remains unclear.

Questions for authors

Why is a separate buffer layer necessary, rather than using the original final Transformer block and LM head as a lens? How would this last-block lens compare with the proposed method in terms of Top-K overlap and multilingual interpretation? Clarifying the unique analytical benefits of the buffer would substantially strengthen the contribution.

Reviewer_DfacOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates the sudden change in representations on the final layer of LLMs, identifies its root cause, and proposes a clever way to mitigate its consequences. Specifically, the authors begin by identifying, through architectural ablations, the exact components responsible for this jump; they show that across models, the pair of linear transformations closest to the output determines where the final-layer jump occurs. Then, the authors propose inserting a "buffer layer" immediatelly before the vocabulary projection, essentially shifting the observed jump outside the normal forward pass of the model. Finally, they show that merging the buffer layer and the LM head after training, and using this as the new vocabulary projection, is able to improve the performance of Logit Lense across several domains.

Critical strengths
  • The paper is very well written and the code appears correct! I had some initial doubts after a brief look over the abstract and introduction and thought there might be a mistake in measuring the quantity of interest, but this is not the case!

  • The experiments are overall very well constructed, the phenomenon is established robustly, its isolation is done properly, and the proposed method is incredible simple and intuitive due to the care the authors took in establishing every step of their work.

  • The authors actually solve the problem they identify and discuss!

Critical weaknesses

I have two main concerns, which are closely related:

  • Contribution and significance: The paper's main contribution seems to be the proposed solution. The "jump" has already been identified in [1], which the authors cite and from which they adopt the definition of displacement, and a non-trivial amount of the analysis overlaps with that work. Specifically, Appendix D of [1] shows that suppressing only the displacement of the final-layer relocates the "jump" to the previous layer, while Appendix K reports improved intermediate decoding under JREG. Similarly, Tuned Lens [2] is an attempt to address a related problem of readout mismatch. Nevertheless, the authors adequately discuss and compare against JREG in Appendix I.1, and their experiments are strong. My concern is that the paper does not sufficiently motivate the significance of solving this particular problem. The interpretability analysis occupies about a page in the main paper (Section 5), with a few additional pages in the appendix, much of which consists of heatmaps. The page count itself is not the really the issue, but I would like a deeper discussion of what we learn or can do given this solution, because otherwise it hard for me to justify the computational cost described below.

  • Computational cost: The proposed solution, while smart and grounded, requires a significant training budget, even at the 1B scale. Training from scratch is prohibitive, and although the authors provide a way to adapt pretrained models, the procedure, as described in Appendix J, uses 2.6B tokens to finetune. Therefore, without a clearer demonstration of what the proposed intervantion enables, it is really hard for me to assess the solution as a worthy contribution.

[1] Shibata, Keigo, et al. "Suppressing Final Layer Hidden State Jumps in Transformer Pretraining." Findings of the Association for Computational Linguistics: EACL 2026. 2026.

[2] Belrose, Nora, et al. "Eliciting latent predictions from transformers with the tuned lens, 2025." URL https://arxiv. org/abs/2303.08112.

Questions for authors

See weaknesses.

I think it will be easy to convince me to raise to weak accept given a sufficient discussion and some more experiments, beyond "showing that the final-layer jump alters the results of Vocabulary Projection Methods and should therefore be taken into account".

Depending on the strength of the additional evidence and discussion provided in the rebuttal, I am also open to raising to a clear accept, but I think it will be a bit hard. I am saying this to give you a prior on my stance, not to discourage you to try :-).

Comments on the paper

2 comments

Thank you for the response and additional experiments. I now understand the role of the buffer much better: it absorbs the jump without increasing the expressive capacity of the readout, yet substantially changes the interpretation obtained from the standard LM-head-only lens. This addresses my main concern.

However, I do not have enough context to judge how important or necessary this analysis and the proposed intervention are, so I think it will be important for the authors to convince the other reviewers on this point.

That said, I find the empirical observation that the jump location is strongly tied to the pair of linear transformations closest to the output quite interesting. I will raise my score to 3.

Thank you for the discussion and for acknowledging the role of the buffer and our empirical findings. The comparison with the last-block lens that you suggested helped us clarify this point.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.