acceptodds
Withdrawn from ICLR 2027

Intermediate Decoder Ladders: Beyond Temperature Calibration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learned readouts of intermediate language-model states are usually fitted to the final layer's full predictive distribution. We test whether this target still lowers the observed next-token negative log-likelihood (NLL) once a competing readout fitted to the teacher's argmax receives its own validation-selected temperature. Holding the intermediate states, frozen head, readout family and search budget fixed, we fit six decoder families on Qwen3-8B and Qwen3.5-9B and fix every fit and temperature before scoring 1,200 additional deduplicated documents. The full target lowers observed NLL by 0.124 and 0.224 nats, with simultaneous 95% intervals excluding zero and all 240 configurations agreeing in sign; per-arm temperature scaling removes 72% and 61% of the uncalibrated gap but not the remainder. A Llama-3.1-8B-Instruct/WikiText replication gives 0.266–0.271 nats across three training seeds, and its full affine readouts keep a 0.56–0.72-nat gain when rescored without refitting on 400 untouched articles, both in the first 128 tokens and in a later window. On Llama, reassigning whole teacher distributions among same-argmax positions removes 74–81% of the gain, and swapping only the tail while keeping each position's argmax probability removes 61–71%. The gain is largest for full affine maps, which stay 0.14–0.16 nats ahead of rank-256 maps under an extended budget, and the fitted readout beats the best calibrated cheap readout by 2.4–2.7 nats. For observed NLL, the full teacher distribution is the better of the two tested targets.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, logit lens, intermediate representations, readout calibration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_5pnCOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

Contribution: The authors aimed to answer the question of whether training an intermediate-layer decoder on a teacher model's full probability distribution produces better observed-token NLL than training on its argmax, even with temperature calibration. Based on my reading, they did indeed answer the question, with the result being positive, across several models and decoder parameterizations. In particular, the authors fit multiple readout families at different transformer depths, keeping the backbone and output head frozen, and compared hard versus soft targets under matched fitting procedures. They tested two Qwen models and a Llama model, evaluated on held-out text, and investigated calibration, readout capacity, and shuffled teacher distributions.

The primary Qwen experiments report NLL improvements of 0.124 and 0.224 nats after calibration. The Llama replication reports approximately 0.27 nats.

The paper's more distinctive finding is that reassigning or swapping non-argmax teacher probabilities reduces the benefit, suggesting that position-specific distributional information matters under the tested fitting procedures.

Critical strengths
  • I believe that the claim was soundly proven, and there are sufficient ablations. The authors conducted controlled comparisons among different architectures, depths, and readouts, and used independent held-out evaluations and many calibration and target controls.
Critical weaknesses
  • Writing quality is very low. Introduction in particular is written in a very confusing and circular way, with sentences winding around. The premise of the paper is not well-articulated, and the core experiments and findings are not motivated strongly.
  • In a sense, I think the outcome of this paper is not one that is particularly strong. The main argument is that soft-target supervision should be expected to outperform argmax supervision on a distribution-sensitive objective (after scalar calibration); the paper does indeed establish this, but fails to articulate why the result is an important advance beyond existing knowledge-distillation literature.
Questions for authors
  • What is the main scientific insight beyond the established advantages of soft-target distillation? In particular, what conclusion about intermediate representations follows from these experiments that would not already be anticipated from the distinction between hard and soft supervision?
  • What practical or scientific decision changes because of these findings? What evidence supports that broader implication?
Reviewer_LXxcOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates whether training an intermediate-layer lens on the final layer’s full distribution, rather than its argmax, still improves observed next-token NLL once each lens receives its own post-hoc temperature. And they consider different lens capacities. As in, scalar, low-rank, or full affine. The paper confirms that the common practice largely remains the optimal one. Mainly, they show that softloss performs better than hardloss, with the gap increasing upon the introduction of a temperature parameter. And this gap is layer dependent. They show that the rank of the lens is important, and that the gap between the softloss and the hardloss largely emerges due to the difference in their predictive distribution, not their top choice for the output. Furthermore, they show that an affine transformation is necessary for the lens.

The paper is very difficult to read, and it is not related to my field of study. And I admit, I was not able to understand the entirety of the content. Consequently, I reduced my confidence score for the review. However, I would like to note that this on its own is grounds for rejection.

Critical strengths
  • The experiments involve learning rate sweeps, which increases my confidence in the results and the findings.
Critical weaknesses
  • The paper is extremely difficult to read. There are variables in the text that are not defined (e.g. H in line 102) and extremely complicated sentences with a lot of numbers and shorthands that are not defined in the paper. I believe a large portion of the text is heavily AI generated. As a result, I had a very hard time understanding the paper.
  • The paper has some unconventional formatting choices (eg. lines 166-173, putting a table in the middle of the text) that makes it difficult to read.
  • The results are presented poorly. Often within text, which makes the paper extremely hard to read.
  • The section starting at line 426 and the results of section 4.2 are trying to answer the same question. The authors should merge these two sections.
Questions for authors
  • Overall, the paper reads like a large report of some experiments, with barely any conclusion present. For example, what's the point of showing that increasing the rank of the lens is helpful to performance? That's completely expected, no? Overall, all I took from the paper is that the standard procedure used for lens training is the right one.
  • I think one important experiment to consider is what would happen if the lens was trained on NLL instead of KL.
Reviewer_tePaOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper explores logit lensing in LLMs through model’s intermediate layers. They explore training these lenses through hard target (learning the teacher’s most likely token) and soft target (learning the full distribution instead) learning. And the exploration is particularly confined at receiving each method’s their own validation-selected temperature. The paper lacks both the necessary motivation for this particular exploration, and a reasonable contribution they intend to make to the field.

Critical strengths

I find no particular strength of this paper, and I explain my reasons in the weakness section.

Critical weaknesses
  • The paper’s central purpose for exploration is weak and doesn’t constitute a valid research problem. The simple exploration for which of soft and hard target learning improves under very specific conditions does not construct a good purpose for a research in my opinion. The authors fail to provide necessary context to such exploration. 
  • The paper describes a number of additional experiments, while failing to construct reasoning and purpose for those experiments as to why they are significant. 
  • The writing of the paper is extremely poor. Instead of developing a research question the authors describe the experiments and results starting from the abstract and introduction sections. This makes it very difficult to understand any context behind the experiments, and after careful reading I failed to find any as well.

Due to these reasons I believe this paper fails to reach the acceptance threshold and require major revision.

Questions for authors

None.

Reviewer_iJ1iOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks whether a frozen-head readout of intermediate language-model states fitted to the teacher's full next-token distribution attains lower observed-token NLL than the same readout fitted to the teacher's argmax, once each arm receives its own validation-selected temperature. Across six readout families on Qwen3-8B and Qwen3.5-9B, and a Llama-3.1-8B-Instruct/WikiText replication with entropy-matched, tail-permuted, shuffled and tail-swapped control targets, the full-distribution target wins by 0.12–0.27 nats on average and by 0.5–0.7 nats for full affine maps, which in turn beat rank-256 maps by about 0.15 nats and cheap calibrated readouts by 2.4–2.7 nats.

Critical strengths

Critical Strengths

The sequence of control targets (entropy-matched, DKPP permutation, stratified whole-distribution shuffle, confidence-preserving tail swap, class-mean tail) is a careful way to ask which property of the teacher distribution a frozen-head readout exploits, and the attainability check of Table 20 removes one confound.

Critical weaknesses

Critical Weaknesses

  1. That a student fitted to the full teacher distribution beats one fitted to the teacher's argmax is the content of Hinton et al. (2015); that the gain is not merely label smoothing is the content of Tang et al. (2020) and of the DKPP experiments of Furlanello et al. (2018), both of which the paper cites. The paper transplants this to frozen-head affine readouts, where the student shares the teacher's output space and, by the paper's own Table 20, the target is reproducible by the head to within 4×10−64\times10^{-6} nats. In this setting the result is less surprising still.

2.Section 2 concedes that no lens study fits an argmax target; Tuned Lens, Low-Rank Lens and OmniLens all minimize KL to the full distribution. The paper therefore confirms existing practice against an arm that is not in use. The title's "beyond temperature calibration" answers a question that was never open: the hard objective is a one-hot cross-entropy that supplies no supervision on non-argmax logits, and a positive scalar preserves every logit ordering, so it cannot recover tail information that the fit never received. Rejecting this null is not informative.

Questions for authors
  1. Please add a readout fitted directly to observed next tokens, and one fitted to a mixture of observed and teacher targets, under the same budget and calibration. How do they compare with the full-target readout on observed NLL?

  2. Why is the equal-family mean the primary estimand? Please report the soft-minus-hard effect for full affine alone on Qwen with simultaneous intervals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.