acceptodds
Under review as a conference paper at ICLR 2027

Pretraining Latent Information Feedback Transformers with Teacher Supervision

Abstract

Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across generation steps is the decoded token. This narrow channel forces models to recompute intermediate results and to discard alternative continuations. In this work, we remove this bottleneck during pretraining, introducing the LIFT (Latent Information Feedback Transformer) architecture and training method which enable LMs to propagate state across generation. We achieve this by turning recurrent-state learning into a teacher-forced prediction problem: each input token is paired with an information-dense state, derived from the next-token distribution of an off-the-shelf pretrained LM. The model, extended with a small number of additional parameters, is then trained to predict both the next token and the next state. As the input states are precomputed, pretraining remains fully parallel across positions. At inference, the model's own predicted states are fed back, with a minor computational overhead that decreases with model size. Experiments with pretrained models ranging from 135M to 1B parameters show that LIFT consistently outperforms standard Transformers and baselines on language modeling, downstream reasoning tasks, and procedural tasks under token-matched budget, while being on par or ahead of compute-matched Transformers. Moreover, a controlled study on a state-tracking task shows that a tiny LIFT outperforms same-size Transformers trained on 8× more data, even when trained with the states of a Transformer that fails the task. Overall, we show that LMs can learn to exploit deep-to-shallow feedback through an efficient and scalable pretraining procedure.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
language models, pretraining, Transformer architecture, deep-to-shallow feedback, recurrent state, latent reasoning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_jDyyOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes LIFT, a Transformer that feeds a "state" — its own top-k next-token distribution — back as an additional input at the next position, giving a deep-to-shallow channel beyond the sampled token. A small fusion layer (~3% of parameters at 1B) merges the embedded state with the token embedding. To keep pretraining parallel, input states are teacher-forced from a frozen pretrained LM run over the same text, and the student is trained with next-token loss plus a top-k forward-KL alignment to the teacher; in the last 10% of training it adapts to its own states, and at inference it feeds back its own predictions. On S5 permutation composition, a 2-layer LIFT solves lengths where same-depth Transformers fail, even with a teacher that itself fails. Pretraining OLMo 2-style models at 135M/350M/1B for 5× Chinchilla tokens, LIFT outperforms token-matched Transformers on perplexity and most downstream metrics, is level with or ahead of compute-matched Transformers (counting the teacher's forward pass), beats a distillation baseline, and compares favorably with Jacobi-iteration feedback models (T2MLR and a reimplementation of the Full-bandwidth Transformer) at matched tokens and compute.

Critical strengths
  1. A simple, scalable way to pretrain a latent feedback channel. Teacher-forcing the fed-back states from an existing pretrained LM keeps training fully parallel, avoiding the approximation-quality vs. training-cost tradeoff of the Jacobi-iteration approaches (T2MLR, Full-bandwidth/Multi-pass Transformer), and is a different route from those works. It works with a same-size teacher (larger teachers do not consistently help; Tables 7–9, 11), and on S5 even a teacher that fails the task suffices, so the requirement on the teacher is mild. Since capable open pretrained models are readily available, treating the teacher as an existing resource is reasonable.

  2. Careful, well-controlled experiments. The paper compares against token- and compute-matched Transformers (with the teacher's forward pass charged), a distillation baseline that isolates the alignment objective (plus tuned distillation variants, App. D.1), the same model run without states, and reproduced feedback baselines, and includes informative ablations (teacher as input vs. target vs. none, channel width, trunk vs. adaptation phase). The S5 experiment cleanly illustrates the expressivity motivation.

Critical weaknesses

1. The reasoning claims rest mainly on likelihood metrics. The abstract claims gains on "downstream reasoning tasks" and "procedural tasks", but at these scales the evidence is largely in bits per byte rather than task accuracy. On the authors' own arithmetic task, exact-match accuracy is roughly 3–6% for every model from three operands on (Table 12), so the reported arithmetic gains are BPB differences on answers no model gets right; pattern continuation is likewise headlined in BPB. Eight of the twenty downstream suites, including GSM8K, MATH, HumanEval and MBPP, are at chance or floor for all models and are excluded from the main table (§5.1, App. B.4). The likelihood improvements are real, but they do not establish improved multi-step reasoning; accuracy-based evidence (e.g., at larger scale, or on tasks where models are above floor) is needed to support those claims.

2. Statistical strength of the matched-compute comparison. Against the compute-matched Transformer the margins are small (e.g., at 1B: PPL 18.81 vs. 19.17, MC 52.5 vs. 51.6, Gen. 35.0 vs. 34.8), and the 350M and 1B models are trained with a single seed; the bootstrap intervals in Table 6 capture evaluation-set variance, not training variance. At 135M, where three seeds are available, most downstream differences at matched compute are within seed noise (App. D.1), and in the §5.3 setting LIFT is level with the compute-matched Transformer on perplexity and slightly behind on MC and RC (Table 14). The token-matched gains are clear; the compute-matched downstream gains would be more convincing with seed variance at larger scale.

Questions for authors
  1. Can you report accuracy-based results that support the reasoning claims, e.g., exact-match arithmetic or pattern continuation at larger scale, or tasks where the models are above floor? Do the BPB gains translate into accuracy anywhere?
  2. Can you report seed variance for the 350M and/or 1B compute-matched comparison (even two additional seeds), so that the downstream margins can be assessed?
  3. Since teacher-forced states and Jacobi-style refinement address the parallel-training problem in different ways, have you considered combining them (e.g., using teacher states as the initialization for refinement passes, or refining after teacher-forced pretraining)? Would they be complementary?
  4. The Transformer baselines are LIFT without the fusion layer, so LIFT has more parameters (~3% at 1B; 138.2M vs. 134.5M in §5.3), and the compute-matched baseline receives extra tokens rather than parameters. The ablations with LIFT's exact parameter count (no-teacher, teacher-as-input-only, k = 1, LIFT w/o states) suggest the extra parameters are not the source of the gain, but a direct parameter-matched control (e.g., a vanilla Transformer widened or deepened to LIFT's parameter count, at matched tokens and compute) would make this explicit. Could you include it?
  5. Does teacher supervision cause the model to use the recurrent path less, i.e., to consume the teacher's (and later its own) states as an informative input rather than learning to carry its own computation through the channel? Could you analyze how the channel is used in the pretrained models, e.g., how much information the fed-back states carry beyond the teacher's next-token distribution, how performance changes when states are perturbed or replaced, or how a small-scale LIFT trained with exact recurrence compares in its reliance on the channel?
3 comments
AuthorsOpenReview ↗

Response to Reviewer jDyy (1/3)

Response to Reviewer jDyy (1/3)

We thank Reviewer jDyy for the careful review. We are glad the reviewer sees LIFT as "a simple, scalable way to pretrain a latent feedback channel", finds treating the teacher as an existing resource reasonable and the experiments careful and well controlled. We ran new experiments to address the reviewer’s questions. All new results and analyses below will be in the revised paper PDF. The full tables and figures are already available in an anonymous repository: https://anonymous.4open.science/r/lift-rebuttal-results-0367/.

W1: The reasoning claims rest mainly on likelihood metrics

We agree that accuracy is the stronger evidence. At these scales, however, exact match is at the floor on most reasoning tasks for every model. We therefore rely on the common practice of using BPB as a proxy for performance in small-scale experiments: OLMo3's evaluation suite uses BPB tasks on math, code and QA for its small-scale experiments, as our RC column does, DataDecide (Magnusson et al., ICML 2025) finds that likelihood metrics at small scale predict rankings at larger scale better than accuracy, and Heineman et al. (NeurIPS 2025) find that BPB has a far higher signal-to-noise ratio than accuracy on GSM8K and MBPP, two of the suites at floor here.

We will revise our reasoning claims accordingly: at matched compute, LIFT's advantage will be stated as a likelihood result. Still, we add accuracy benchmarks to the main text to better support the reasoning claim:

1. Pattern continuation, with a new accuracy metric: This task (OLMES, 534 questions) asks the model to continue a sequence by a rule shown in context (e.g., 2 4 6 8 → 10; @ # % @ # → %). Besides the multiple-choice (MC) accuracy already in App. C.6, we measured the exact-match (EM) accuracy of the generated continuation. LIFT has the highest EM of the Table 1 models at every scale, +4.5 / +5.9 / +5.3 points over the token-matched Transformer at 135M / 350M / 1B (Table R2a), consistent with the best BPB results we report in the main text. MC accuracy agrees: LIFT is the highest of the Table 1 models at every scale (tied at 135M) and ahead of the token-matched Transformer in all 7 independently trained pairs (3 seeds for 135M and 350M, 1 seed for 1B) on both metrics. At 1B it also beats its teacher, trained on 2× the tokens (Tables 10–11 in the anonymous repository). Following the reviewer's concern, we will move these accuracy results (existing and new) from the appendix (the paper's Table 13) to the main text, with three seeds at 135M and 350M.

Table R2a. Pattern continuation, generative exact match (%). 135M and 350M: mean ± sd over three seeds, 1B and the teacher: one run.

Model135M350M1B
LIFT26.2 ± 1.137.9 ± 1.750.9
LIFT w/o states25.0 ± 1.035.8 ± 4.944.2
Transformer, same tokens21.7 ± 2.031.9 ± 1.645.6
Transformer, same compute24.6 ± 0.837.2 ± 1.045.6
Distillation24.2 ± 1.536.7 ± 1.547.6
Teacher (Transformer, 2× tokens)26.337.449.1

2. New benchmark: S5 state tracking at LM scale. We fine-tuned all the pretrained models in Table 1 on the S5 task of Sec. 4 (8,000 steps on 1–16 permutations, loss on the state after every step). LIFT is fine-tuned with the paper's training recipe: its input states come from a Transformer baseline fine-tuned the same way, with the alignment loss. LIFT is exact at every length up to N=64, although its teacher falls to chance from N=32, (Table R3). The results are the same at 135M and 350M (Table 13 in the anonymous repository).

Table R3. S5 accuracy (%) of the fine-tuned 1B models on sequences of length N; chance is 0.8%.

ModelN=16N=20N=24N=32N=64
LIFT100100100100100
Distillation99.458.15.11.20.9
Teacher (Transformer, 2× tokens)99.860.43.21.10.9
AuthorsOpenReview ↗

Response to Reviewer jDyy (2/3)

Response to Reviewer jDyy (2/3)

W2 + Q2: Seed variance at larger scale

New: we trained two more seeds at 350M, which now has three seeds for every model of Table 1.

Table R6. 350M, mean ± sd over three seeds.

ModelPPL ↓MC %Gen. %RC BPB ↓Arith. BPB ↓
LIFT23.82 ± 0.0144.6 ± 0.423.8 ± 0.70.921 ± 0.0012.13 ± 0.03
LIFT w/o states24.98 ± 0.0443.9 ± 0.321.8 ± 1.10.944 ± 0.0012.23 ± 0.01
Transformer, same tokens25.14 ± 0.0243.5 ± 0.521.3 ± 0.90.959 ± 0.0032.31 ± 0.14
Transformer, same compute24.30 ± 0.0244.5 ± 0.422.7 ± 0.60.938 ± 0.0062.26 ± 0.06
Distillation24.53 ± 0.0144.2 ± 0.222.4 ± 0.50.936 ± 0.0042.21 ± 0.07

The three seeds at 135M (the paper's Table 10) and 350M (Table R6), with the 1B run (the paper's Table 1), give 7 independently trained pairs of LIFT and the compute-matched Transformer. LIFT is ahead:

  • on perplexity, arithmetic, Gen. and RC in all 7 pairs (Table 9 in the anonymous repository);
  • on MC in only 4 of 7 pairs, so at matched compute MC is level (44.6 ± 0.4 vs 44.5 ± 0.4 at 350M), and the revised §5.2 will say so;
  • on 61 of 84 individual suite comparisons. Table 1's MC, Gen. and RC columns are means over 12 suites (App. B.4). Comparing the two models on each suite separately, in each of the 7 pairs, gives 84 comparisons; LIFT is ahead in 15 of 21 on the MC suites, 12 of 21 on the generative suites and 34 of 42 on the BPB suites.

The perplexity lead is many times its seed spread: 0.47 ± 0.08 at 135M, 0.50 / 0.48 / 0.45 in the three 350M seeds, and 0.36 at 1B. In tokens, the Transformer needs 1.6× / 2.1× / 2.0× LIFT's tokens to reach its loss at 135M / 350M / 1B, while LIFT's training compute per token is 1.42–1.44× a Transformer's.

Q1. Gains in terms of accuracy

Yes, on pattern continuation, where BPB and accuracy are measured on the same questions (W1, point 1). LIFT's BPB lead comes with the highest generative EM (Table R2a) and the highest MC accuracy (Table 11 in the anonymous repository; tied at 135M) of the Table 1 models at every scale.

Q3. Combining teacher-forced states with Jacobi-style refinement

We believe these methods are complementary, and LIFT already combines them in a simple form: its adaptation phase (App. A.3) drops the teacher and trains on the model's own states, produced by one or two parallel passes, i.e., refinement after teacher-forced pretraining. The ablations show the two add up (App. D.1, D.3): refinement alone, without a teacher, only ties the compute-matched Transformer at 135M; at 1B the teacher-forced trunk alone keeps 73% of LIFT's gain, and the refinement phase adds the rest. The fuller combination the reviewer suggests, the teacher as a bootstrap followed by multi-pass Jacobi refinement, is a direction we are actively exploring.

AuthorsOpenReview ↗

Response to Reviewer jDyy (3/3)

Response to Reviewer jDyy (3/3)

Q4. A direct parameter-matched control

We conducted the experiment the reviewer suggested. At 135M the fusion layer is a larger share of LIFT's parameters (4.6%, against 3% at 1B), so this is where extra parameters could matter most. We widened the Transformer's MLP (ratio 9 instead of 8) to 249.6M parameters, against LIFT's 248.5M; its added FLOPs per token match LIFT's (the fusion layer plus the k = 1024 state embedding). We trained it with Table 1's seed and data order at matched tokens (13.4B) and at matched compute (Table R8; also Table 14 in the anonymous repository).

Table R8. 135M, seed of Table 1.

ModelParamsTokensPPL ↓Arith. BPB ↓MC %Gen. %RC BPB ↓
Transformer237.0M13.4B32.432.4638.113.61.110
Transformer, parameter-matched249.6M13.4B32.082.4738.513.31.106
Transformer, parameter-matched249.6M17.9B30.992.4639.512.51.079
LIFT248.5M13.4B30.652.2939.415.51.078

At matched tokens, the extra parameters give the Transformer 0.35 PPL, a fifth of LIFT's gain (1.78). At matched compute, LIFT is 0.34 PPL ahead of the parameter-matched Transformer and clearly ahead on arithmetic and Gen.; MC and RC are level. Together with the exact-parameter ablations the reviewer notes, this shows that the gain does not come from the added parameters.

Q5. Teacher supervision and recurrent path utilization

It is challenging to separate “informative input” from “carrying its own computation” (the model's own computation is itself an informative input), but here are results from our analyses of these questions:

  • Exact recurrence vs. state teacher forcing (the S5 models of Sec. 4). LIFT-RNN is trained with full backpropagation through the recurrence, so it has an explicit signal to put relevant computation on the state. Its teacher-forced students never receive this signal in training, yet they inherit this property from the RNN teacher and extrapolate beyond the training lengths.
  • Perturbing states in the pretrained LMs (135M / 1B). Switching the channel off costs 1.8 / 0.8 PPL, and another sequence's states are far worse than none (52.3 vs 31.6 PPL at 135M). The top 256 tokens keep 89% of the gain, while a one-hot of the top token (an input never seen in training) is worse than no state (Figure 5 in the anonymous repository plots the C4 perplexity of every state variant against switching the channel off; Table 16 lists the values).

We thank the reviewer again, and hope our response resolves their concerns.

Reviewer_QSHNOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Authors propose LIFT, an LM architecture that enables state tracking across different token states that uses latent recurrence across different states; specifically, past tokens' prediction heads jointly predict next-token distributions and a unconstrained state, which is looped into the input of the next token during autoregression. Unlike prior work, authors uses a pretrained transformer as the source of the state during training, whereby they supervise the model under training to predict the state of this pretrained transformer. THe authors demonstrate improved state-tracking behavior across S5 permutations, as well as gains in arthmetic and language modeling in FLOPs and token matched settings.

Critical strengths
  • the authors get the benefits of state tracking without needing to incur expensive sequential training costs
  • the authors demonstrate superior performance on BPB, S5 permutation, and arithmetic tasks across 135M-1B settings
  • the authors also demonstrate token efficiency to solve tasks using this model, highlighting the effectiveness of state passthrough; although this has been shown in similar works, the authors achieved it without needing expensive self-supervised method that adds huge training overheads
Critical weaknesses
  • I would've enjoyed seeing what other things can be packed into the states than mere demonstrations (see my point below about this being a form of distillation)
  • Similar to the point above would be interesting to understand the relationship between the quality and size of the supervising model and the size of the model being trained, including whether the system works with a different tokenizer or on a differently-trained text distribution.
  • A potential understanding of the system can be that its gains partly originate a more information-rich form of distillation where supervision extends beyond posterior distributions towards the structure of the residual streams instead. Disentangling this effect could be helpful to understand where the gains come from.
Questions for authors
  • As the authors suggest, it would be really interesting to see what exactly is the relationship between varying the model that is used to perform supervision and the model under training along architecture and especially tokenization/vocabulary. Do the authors believe their gains originate from simply a "richer" token embedding, or from something more fundamental that can potentially help reduce the extra supervising-model inference compute during training as well?
  • Similarly, to disentangle the effects of the richer residual structure supervision, has the authors tried seeing what the performance of a baseline looks like when distilling raw residual states rather than posterior token distributions?
  • Has the authors done any sort of interp, logit lens, probing analysis on the additional states the end up learning?
4 comments
AuthorsOpenReview ↗

Response to Reviewer QSHN (1/3)

Response to Reviewer QSHN (1/3)

We thank the reviewer for the thoughtful and positive review. To address the points raised by the reviewer, we conducted a new analysis of what the states carry (W1, Q3) and new experiments on the teacher's role (W2, W3), and we share results from our early exploration with hidden states (Q2). All new results and analyses below will be in the revised paper PDF. The full tables and figures are already available in an anonymous repository: https://anonymous.4open.science/r/lift-rebuttal-results-0367/.

W1: What can be packed into the states

We analyzed the states of the trained LIFT students in Sec. 4, which differ only in their teacher (the recurrently trained LIFT-RNN or a Transformer) and seed. Interestingly, we observe they encode the intermediate result (the permutation composed so far) in three different ways (example states are in Figure 4 in the anonymous repository):

  1. One-hot. Almost all of the state's mass is on the intermediate result. This is the code with the LIFT-RNN teacher, in 2 of 3 seeds.
  2. Soft code. The state is spread around the right answer: for example, 8% on the intermediate result, with the next most likely permutations one swap away from it. The model is unsure, but using its "best guess" is enough to solve the task up to 12 steps (more than a same-depth Transformer can). This is the code with the Transformer teacher, in all 3 seeds.
  3. Opaque code. The most surprising one, used by the LIFT-RNN teacher of the third seed and inherited by its student. The model encodes the intermediate result in other, unrelated tokens, such as unused number tokens: in 61% of steps the correct permutation is not the top token, and in the example of Figure 4 it is not in the state at all. Yet the model solves the task (100% up to 16 steps, and 98.9% at 24 steps). A linear probe, fitted on separate sequences, reads the intermediate result from these states with 99.6% accuracy (chance: 0.8%).

This matters for the interpretability of latent reasoning: a state made of token probabilities, which looks easy to read, can learn to carry information in a hidden form, at least when, as for the LIFT-RNN, the states carry no direct loss. Q3 adds how the states can be steered, why the full distribution matters, and where LIFT computes each step.

W2: Teacher quality and size, tokenizer, and training data

Teacher quality and size. To better understand the connection between the teacher and the student, we report LM results with public teacher checkpoints of much larger size or training budget: the OLMo 2 1B checkpoint, trained on 4T tokens, and the OLMo 2 7B checkpoint, both as teachers of our 1B LIFT (Table R9). They also differ from our teachers in their training data, though it is similar in distribution. LIFT is highly robust to the teacher's size and training budget: teachers up to 4.9 PPL stronger change LIFT's perplexity by at most 0.25, and by only 0.04 at 350M and 1B (Table R9).

Table R9. C4 perplexity of LIFT with each teacher.

StudentTeacherTeacher PPLLIFT PPL
135M135M, ours29.8030.65
135M350M, ours24.9130.40
350M350M, ours23.4923.82
350M1B, ours19.7223.78
1B1B, ours18.6218.81
1BOLMo 2 1B, public17.3118.77
1BOLMo 2 7B, public14.9718.85

These results suggest that the teacher’s states only need to be informative enough for LIFT to learn to use its feedback input. A more accurate teacher adds little beyond that, because at inference LIFT feeds back its own states, not the teacher’s.

Tokenizer. The teacher and student must share a vocabulary, but not a specific one: our main experiments use OLMo 2's (100K tokens), and Sec. 5.3 uses SmolLM2's (49K). We believe LIFT can be trained with a teacher that uses a different tokenizer, but this would first require adapting cross-tokenizer distillation methods (Boizard et al., TMLR 2025; Minixhofer et al., NeurIPS 2025) to map the teacher's distribution to the student's tokens. This is an interesting direction for future work.

AuthorsOpenReview ↗

Response to Reviewer QSHN (2/3)

Response to Reviewer QSHN (2/3)

W3: Is the gain a richer form of distillation that extends to the residual stream?

This is an interesting question, and we believe the answer is no. The clearest evidence is a new experiment showing that LIFT’s gains persist when the knowledge signal fades: we trained LIFT to its teacher’s own budget (29.6B tokens) and past it, so the teacher becomes weaker than the LIFT student. If LIFT mainly benefited from a richer form of distillation rather than from the feedback channel, we would expect its gains to fade as the teacher becomes weaker. Distillation’s gain over the Transformer indeed falls from 0.62 PPL to zero, but LIFT stays 1.1–1.2 PPL ahead of Distillation and keeps a steady token multiplier over the Transformer, over three seeds (Table R4 below; Figures 1–2 in the anonymous repository plot the training curves). So the gain comes from the feedback channel, not from a richer form of distillation.

Table R4. 135M, C4 perplexity, mean ± sd over three seeds

LIFT tokens (vs. teacher's)Transformer, same tokensTransformer, same computeDistillationLIFTDistillation's gain over TransformerLIFT's lead over Distillation
13.4B (0.45×, Table 1)32.44 ± 0.0231.04 ± 0.0031.81 ± 0.0430.57 ± 0.080.62 ± 0.031.24 ± 0.11
29.6B (1.0×)29.83 ± 0.0529.05 ± 0.0429.71 ± 0.0228.58 ± 0.050.12 ± 0.041.13 ± 0.07
43.0B (1.45×)28.95 ± 0.0528.31 ± 0.0429.01 ± 0.0327.92 ± 0.03−0.06 ± 0.021.09 ± 0.05

Q1: Relations between teacher and student, gains come from a "richer" token embedding?

Teachers of other sizes and tokenizers are discussed in W2. We see LIFT's gain as more fundamental than a "richer" embedding: it widens the Transformer's information bottleneck, letting each position pass more processed information to the next than a single token. The full output distribution of a pretrained Transformer is the state we found to work (Q2 compares the hidden state, and Q3 shows that the whole distribution carries information). But the LIFT recipe, teacher-forced states for parallel training, is not tied to it: it can use non-Transformer teachers (Sec. 4 shows LIFT inheriting an RNN's sequential state passing) or other kinds of states (e.g. more efficient teachers could reduce its inference overhead in training).

Q2: A baseline that distills the teacher's hidden states?

Yes, it was our first design: we used the teacher's hidden state (residual stream) as the state, both as a distillation target and as the fed-back input (Table R10; an earlier recipe, so not directly comparable to the paper's numbers).

Table R10. Hidden state vs. output distribution as the teacher signal (135M, C4 perplexity).

ModelPPL, teacher's states fedPPL at inference
Distillation of the hidden state–31.95
Distillation of the distribution–32.10
Hidden state fed back30.3333.84
Distribution fed back (early LIFT)30.3331.07

As a distillation target, the hidden state is slightly better than the distribution, and as an input it is just as informative: given the teacher's states, both feedback models reach the same perplexity. The difference appears at inference, when the model feeds back its own states. Imitating another model's hidden vectors is hard (they have high variance, and small errors add up over positions), so the hidden-state model loses 3.51 PPL on its own states, while the distribution model (what became LIFT) loses only 0.74 (0.32 with LIFT's full recipe; App. D.1). Larger prediction heads and other losses did not close this gap, but we think closing it, so that LIFT can feed back hidden states, is an interesting direction.

AuthorsOpenReview ↗

Response to Reviewer QSHN (3/3)

Response to Reviewer QSHN (3/3)

Q3: Interpretability analysis of the learned states

Yes, we ran a new mechanistic analysis of the states, with linear probes, causal edits of the states, and the logit lens. It covers the 2-layer S5 models of Sec. 4, the pretrained 135M models fine-tuned on S5, and the pretrained 135M and 1B LMs on text. We asked four questions:

  1. What is in a state? The 2-layer S5 models store the intermediate result in one of three codes, depending on their teacher and seed: one-hot, a soft code spread around the right answer, or an opaque code in unrelated tokens that a linear probe still reads at 99.6% (See W1 for more details).
  2. Can we steer the states? We replaced a fed state with a synthetic one: the model's average state for a permutation chosen so that the answer becomes a new random target. In the 2-layer S5 models, editing one state sets the answer to the target in 100% of sequences when the model uses the one-hot code, and in 86–99% with the soft code. With the opaque code, one edit cannot steer consistently (32% for the LIFT-RNN, 75% for its student): the model reads this code from several recent states, so a few consecutive edits are needed (99–100% with up to six; Figure 3 in the anonymous repository). So the states can be steered, and hard-to-read codes make steering techniques for latent states an interesting direction to explore.
  3. Does the full distribution matter? Yes, even its tail. We tested whether the information LIFT passes forward lies only in its top prediction. At inference, we fed each pretrained LIFT model reduced versions of its own state and measured C4 perplexity (Figure 5 in the anonymous repository). Compared with no state, the full state (its top 1024 tokens) improves perplexity by 1.75 at 135M and 0.80 at 1B. Its top 256 tokens keep 89% of this gain, its top 64 about half (59% at 135M, 43% at 1B), and its top 16 none of it; a one-hot of the top token, or the same 1024 tokens with equal or shuffled probabilities, is worse than no state at all. So the model relies on the whole distribution it was trained with, down to its tail (ranks 257–1024 hold 11% of the gain). Our ablation of the state width in training (App. D.2) shows that a wider state also carries more: with a state of only the teacher's top token, LIFT keeps 13% of its gain over the token-matched Transformer, with the top 16 tokens 56%, and with 128 tokens 86%, and going from 128 to 1024 tokens still improves perplexity by 0.24 (95% CI 0.20–0.29).
  4. Where do LIFT and the Transformer compute each step? We applied the logit lens (the model's output layer, applied after every layer) to the 135M models fine-tuned on S5, and recorded the first layer at which the new intermediate result (after composing the current permutation) is the top prediction. In LIFT, it is on top right after the fusion layer, before any Transformer block, at every step up to 64: the fusion layer itself computes each step. In the Transformer, it appears one layer deeper every few steps, reaches the last layer (layer 8) by step 16, and never appears after step 18–20 (Figure 6 in the anonymous repository).

We thank the reviewer again for the insightful questions and will be glad to clarify anything further.

Thank you for the responses. The distillation sweep is insightful. I remain positive about the work.

Reviewer_84tBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LIFT adds a feedback channel to a decoder-only LM. Each input position receives both the token and a "state": the previous position's next-token distribution, truncated to the top 1024 tokens, softened with τ=1.5 and embedded as a probability-weighted mix of token embeddings. A small SwiGLU fusion layer merges the state with the token embedding, adding about 3% parameters at 1B. To keep pretraining parallel, the training states come from a frozen teacher LM and are teacher-forced, and an extra KL term pulls the student's predictions toward the teacher's. In the last 10% of training the model adapts to its own states, and at inference it feeds back its own predictions.

On S5 permutation tracking, 2-layer LIFT models solve lengths that a same-depth Transformer cannot. On OLMo-2-style models at 135M/350M/1B trained for 5× Chinchilla, LIFT lowers C4 perplexity by 5–5.5% versus token-matched Transformers (e.g., 18.81 vs 19.79 at 1B) and is ahead of compute-matched ones. At 135M it also compares favorably with Jacobi-style feedback models (T2MLR, multi-pass).

Critical strengths
  1. A simple way to pretrain a feedback model. Teacher-forcing the state channel avoids sequential rollouts and Jacobi iterations, and Table 2 shows that this matters: at matched compute both Jacobi-style baselines fall behind a plain Transformer on every metric except arithmetic, while LIFT stays level with it. On S5, LIFT trained in parallel on the states of a recurrent teacher still extrapolates to N=32 (96%), which is clear evidence that the channel can add effective depth.

  2. Careful and candid evaluation. The paper has token-matched and compute-matched baselines, a distillation baseline, a no-state mode, three seeds at 135M and full per-suite tables, and it reports an ablation that cuts against its own story (below).

Critical weaknesses
  1. Most of LIFT's gain over compute-matched Transformers comes from a stronger teacher whose cost is not counted.

    • In the main experiments the teacher is a same-size model trained on about 2× the student's tokens (29.6B/77.8B/222B vs 13.4B/34.9B/107.4B). The compute match counts the teacher's forward passes but not its pretraining (F.1). The paper treats the teacher as off-the-shelf, "pre-invested" compute, but then the practical question is why one would deploy LIFT rather than the same-size teacher.
    • Without a teacher, LIFT ties the compute-matched Transformer (+0.01 ± 0.05 PPL, D.1); in the authors' words, the teacher "accounts for 98% of LIFT's gain over the compute-matched Transformer".
    • In the Section 5.3 comparison the teacher (B.3) is the compute-matched vanilla model itself (23B tokens). LIFT is level with it on perplexity and MC (18.04 vs 18.02, 38.0 vs 38.1) and better on generation and arithmetic (9.5 vs 8.5, 2.18 vs 2.30).
    • At 1B, the teacher beats LIFT on 10 of the 12 suites in Table 9 (e.g., TriviaQA 41.2 vs 37.3), although LIFT beats it on arithmetic at every scale.

    The fair comparison is LIFT vs distillation from the same teacher. There LIFT wins clearly on perplexity (18.81 vs 19.40 at 1B) but only by small margins downstream (MC 52.5 vs 52.0, Gen 35.0 vs 34.8). The paper also trains LIFT with larger teachers (B.2, Tables 11–12), which is the setting where deploying a smaller model makes sense, but distillation from those teachers is not reported.

  2. Feedback is not separated from knowledge transfer at LM scale, and the claims about procedural tasks go beyond the data.

    • No LM-scale run uses a teacher without extra tokens or compute.
    • On the arithmetic task with 4 or more operands, exact-match accuracy is at most 6.2% for every model (Table 12), so the BPB differences say little about multi-step computation. At 1B, plain distillation beats LIFT on arithmetic (BPB 1.95 vs 1.99; accuracy 6.4% vs 5.9%), but Fig. 3a leaves distillation out.
    • On pattern continuation at 135M, accuracy is identical with and without states (53.2 ± 0.7 in both cases, Table 10).
    • The 350M and 1B results are single runs.
Questions for authors
  1. At 135M with three seeds, please train LIFT with a teacher that has no extra tokens or compute, e.g. the token-matched Transformer itself or a smaller model. Does LIFT still beat both its teacher and the compute-matched baseline?
  2. Please add the teacher's own perplexity to Table 1, plus a compute column that includes the teacher's pretraining. For the larger-teacher runs (B.2), how does LIFT compare with distillation from the same teacher on perplexity and the downstream suites?
  3. Fig. 3b's multiplier reaches 1.6×/2.1×/2.0× at 4× Chinchilla, where the teacher has seen about 2.6–2.8× as many tokens as LIFT. Is the growth simply LIFT approaching its teacher? Adding the teacher's loss to Fig. 3b would answer this.
  4. Please add seeds at 350M/1B.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 84tB (1/3)

Response to Reviewer 84tB (1/3)

We thank the reviewer for the constructive review. We are glad they see LIFT as a simple way to pretrain a feedback model and find the evaluation careful and candid. We ran the experiments the reviewer requested and additional ones, and the results support the paper's claims. All new results and readings below will be in the revised paper PDF. The full tables and figures are already available in an anonymous repository: https://anonymous.4open.science/r/lift-rebuttal-results-0367/.

W1: Using a stronger teacher

The reviewer raises two complementary concerns: (a) In which cases one should use LIFT rather than the teacher it is trained with, and (b) not counting the teacher pretraining in the compute-matched baseline. We address both concerns:

Why not use the teacher instead of LIFT: Thanks for the great question. We see three reasons to prefer LIFT, and the first two are supported by new results the reviewer asked for:

1. LIFT significantly beats its teacher when trained on the same number of tokens: We trained LIFT (3 seeds) at 135M on its teacher's own budget of 29.6B tokens. LIFT beats its teacher in perplexity (28.58 ± 0.05 vs 29.80) and on all downstream columns of Table 1, in all three seeds. In the paper's Tables 7–9, the teacher was ahead only because it was trained on 2.2× more tokens. Full results in Table R1 (more in Q1; for a more detailed graph view, see the extended Fig. 3b in the anonymous repository, Figures 1–2):

Table R1. 135M at 29.6B tokens, mean ± sd over 3 seeds (the teacher is a single model, one of the three token-matched Transformers).

ModelPPL ↓Arith. BPB ↓MC %Gen. %RC BPB ↓
Teacher (Transformer, 29.6B)29.802.39440.115.01.057
Transformer, token-matched (29.6B)29.83 ± 0.052.310 ± 0.07940.0 ± 0.114.8 ± 0.91.056 ± 0.001
Transformer, compute-matched (41.5B)29.05 ± 0.042.359 ± 0.11140.6 ± 0.315.2 ± 0.71.038 ± 0.004
Distillation29.71 ± 0.022.356 ± 0.11440.1 ± 0.115.1 ± 0.41.052 ± 0.004
LIFT28.58 ± 0.052.281 ± 0.04341.0 ± 0.317.4 ± 0.31.033 ± 0.002

2. Deploying a smaller model: The reviewer noted that one setting where deploying LIFT makes sense is when a model smaller than the teacher is needed, but that the comparison that would support it, against distillation from a stronger teacher, was not reported. We now report it (Table R5, Q2): with the same 350M teacher, the 135M LIFT student beats the distilled student in every column, so LIFT is the better alternative.

3. LIFT is more expressive: LIFT has a feedback channel, while its teacher does not. Some tasks, for example those that require state tracking, cannot be solved by regular Transformers of a fixed depth. Our experiment in Section 4 demonstrates this, showing that LIFT solves tasks that regular Transformers (including its own teacher) cannot. To further support this, we fine-tuned all the pretrained models in Table 1 on the S5 task and found that these results hold at LM scale (see W2).

Not counting the teacher "pre-invested" compute:

This concern is about the compute-matched Transformer baseline. Its budget includes all the compute of using the teacher during LIFT's training, but not the teacher's own pretraining. This follows our setting, which assumes a readily available pretrained teacher, an assumption we argue is very reasonable: countless pretrained models of various scales are freely available, and research on how to use such models as a resource for training new ones is interesting, relevant, and of high practical value. It is also the standard accounting in other lines of research, such as knowledge distillation and model merging, where pretraining of the teacher/merged models is not charged. In our own experiments, the teachers are checkpoints taken from our Transformer baselines, so practically we didn't invest more compute to pretrain them. LIFT also works with off-the-shelf teachers: at 1B, with the public OLMo 2 1B and 7B checkpoints from Hugging Face, it reaches the same perplexity as with our own teacher (18.77 / 18.85 vs 18.81, against 19.17 for the compute-matched Transformer; Table 6 in the anonymous repository).

AuthorsOpenReview ↗

Response to Reviewer 84tB (2/3)

Response to Reviewer 84tB (2/3)

W2: Separating feedback gains from knowledge transfer

Is LIFT's gain knowledge transfer from the teacher, or feedback? Three comparisons separate the two (all will be added to the revision):

  1. Distillation baselines: Distillation uses the same teacher only as a target, so its gain over the Transformer measures knowledge transfer, and LIFT's gain over Distillation measures the feedback gain. LIFT is ahead of Distillation in perplexity at every scale (Table 1) and in 30 of the 36 downstream suite comparisons (Tables 7–9). We strengthen this comparison with new results:
    1. We added 350M seeds for LIFT and Distillation (Table R6, see Q4): LIFT consistently beats Distillation (by 0.71 ± 0.02 PPL). Across all 7 independently trained pairs (three seeds at 135M, three at 350M, and 1B), LIFT is ahead of Distillation in perplexity, MC, Gen. and RC.
    2. We added distillation from a larger teacher (Table R5, see Q2): LIFT beats Distillation by 0.91 PPL.
  2. A token-matched teacher: We trained LIFT and all baselines (3 seeds each) on the teacher's own token budget, and then 1.45× past it, removing the teacher's data advantage (Table R4, see Q1). Distillation's gain over the Transformer drops from 0.62 to 0.12 PPL, and to zero past the teacher's budget, while LIFT's lead over Distillation stays at 1.1–1.2 PPL at every budget. Once the knowledge signal is gone, Distillation's gain disappears, but LIFT's gain remains: this is the feedback channel's contribution. We thank the reviewer for suggesting this experiment.
  3. Comparing LIFT with and without the feedback channel: LIFT is trained so that its feedback channel can be turned off at inference, which isolates the feedback gain. Without states, LIFT loses 1.8 / 1.2 / 0.8 PPL at 135M / 350M / 1B (Table 1, Table R6).

W2: Procedural task claims

We strengthened the procedural results in three ways:

1. A generative metric for pattern continuation. Pattern continuation (OLMES, 534 questions) asks the model to continue a sequence by a rule shown in context (e.g., 2 4 6 8 → 10; @ # % @ # → %). We extend the multiple-choice (MC) accuracy and BPB results of App. C.6 with exact-match (EM) accuracy of the generated continuation, and find that LIFT's likelihood lead translates into a generative lead. LIFT has the highest EM of all baselines at every scale (+4.5 / +5.9 / +5.3 points over the token-matched Transformer), and matches or beats its 2×-token teacher, consistent with its lead in BPB and MC (Table R2a).

Table R2a. Pattern continuation, generative exact match (%). 135M and 350M: mean ± sd over three seeds; 1B and the teacher: one run.

Model135M350M1B
LIFT26.2 ± 1.137.9 ± 1.750.9
LIFT w/o states25.0 ± 1.035.8 ± 4.944.2
Transformer, same tokens21.7 ± 2.031.9 ± 1.645.6
Transformer, same compute24.6 ± 0.837.2 ± 1.045.6
Distillation24.2 ± 1.536.7 ± 1.547.6
Teacher (Transformer, 2× tokens)26.337.449.1

2. Seeds at 350M for both procedural benchmarks. With three seeds of every model at 350M (Table R6, Q4), LIFT is ahead of every baseline in pattern MC, pattern BPB and arithmetic BPB (62.7%, 1.124 and 2.13, against at best 61.2%, 1.143 and 2.21).

3. A new benchmark: S5 state tracking at LM scale. We fine-tuned all the pretrained models in Table 1 on the S5 task of Sec. 4 on lengths 1–16, LIFT with the paper's recipe (states from a Transformer fine-tuned the same way, near-exact on these lengths, unlike Sec. 4's 2-layer teacher). LIFT is exact at every length up to N=64, while every Transformer is at chance from N=32, including its teacher, and so does Distillation, at all three sizes (Table 13 in the anonymous repository).

We believe these results now support the procedural claims, and we will move them to the main text (with the pattern-continuation table of App. C.6).

With the new results, the specific numbers the reviewer points to are exceptions rather than the overall trend:

  • "Pattern accuracy at 135M is identical with and without states": True at this one point, but the feedback channel helps in every other procedural comparison: pattern MC at 350M and 1B, and pattern EM, pattern BPB and arithmetic BPB at every scale.
  • "At 1B, plain distillation beats LIFT on arithmetic (1.95 vs 1.99)": A small gap in a single run. With three seeds at 135M and 350M, LIFT is ahead of Distillation in every seed.
  • "With 4 or more operands, exact match is at most 6.2% for every model": Expected at this scale, that's why we report BPB for arithmetic. Generative evidence above the floor comes from pattern EM and S5.

Regarding adding Distillation to the arithmetic results (Fig. 3a): the results of all baselines, Distillation included, for every number of operands and with seeds, will appear in a table in the revision (extending Tables 11–12).

AuthorsOpenReview ↗

Response to Reviewer 84tB (3/3)

Response to Reviewer 84tB (3/3)

Q1: Does LIFT beat its teacher and the compute-matched Transformer when trained to the teacher's budget?

Yes, on both counts, and significantly: far beyond the seed spread at every budget (see also W1 and W2).

New experiment, training LIFT past its teacher's budget: With the paper's 135M teacher (a Transformer trained on 29.6B tokens), we train 3 seeds each of LIFT, Distillation and both Transformer baselines on the same 29.6B tokens, so the teacher has no token or compute advantage. We then continue to 43.0B tokens (1.45× the teacher's budget). At both budgets, LIFT beats its teacher, the compute-matched Transformer and Distillation, all beyond the seed spread (Table R4).

Table R4. 135M, C4 perplexity, mean ± sd over three seeds; each model annealed at its budget. "Same compute" = 1.4× LIFT's tokens (Table 1, App. F.1).

LIFT tokens (vs. teacher's)Transformer, same tokensTransformer, same computeDistillationLIFTDistillation's gain over TransformerLIFT's lead over Distillation
13.4B (0.45×, Table 1)32.44 ± 0.0231.04 ± 0.0031.81 ± 0.0430.57 ± 0.080.62 ± 0.031.24 ± 0.11
29.6B (1.0×)29.83 ± 0.0529.05 ± 0.0429.71 ± 0.0228.58 ± 0.050.12 ± 0.041.13 ± 0.07
43.0B (1.45×)28.95 ± 0.0528.31 ± 0.0429.01 ± 0.0327.92 ± 0.03−0.06 ± 0.021.09 ± 0.05

Q2: Teacher perplexity, and distillation from larger teachers

We ran the comparison the reviewer requested, distilling the larger teacher from App. B.2 (350M, 38.4B tokens) into a 135M student at 13.4B tokens, and compared it with LIFT trained with the same teacher (Table R5). LIFT is ahead in every column.

Table R5. 135M students at 13.4B tokens, with the 350M teacher (one seed).

ModelPPL ↓Arith. BPB ↓MC %Gen. %RC BPB ↓
Transformer, same tokens (no teacher)32.432.4638.113.61.110
Distillation from the 350M teacher31.312.4639.313.11.083
LIFT with the 350M teacher30.402.2939.516.01.066

Regarding adding the teacher's results: we will add a new appendix table with the perplexity and training tokens of every teacher used in the paper (e.g., 29.80 / 23.49 / 18.62 PPL for the 135M / 350M / 1B teachers of Table 1).

Q3: Is the growing multiplier of Fig. 3b simply LIFT approaching its teacher?

No, and the new results in Table R4 show exactly that. If LIFT were only approaching its teacher, the multiplier could not exceed the teacher-to-LIFT token ratio: 1.0× at 29.6B and 0.69× at 43.0B. The measured multiplier is 1.70 ± 0.06× at 13.4B, 1.77 ± 0.08× at 29.6B and 1.78 ± 0.07× at 43.0B (three seeds; no data is repeated: the 135M runs read an 83B-token selection, and App. B.2's "40B" is a typo). We concede one point: past the paper's budget, the multiplier stops growing and holds at 1.7–1.8×, and we will revise the sentence that says it grows. We will also add an extended version of Fig. 3b for 135M, with the teacher's loss, the continued training to 43.0B, and seed error bars. This extended figure is already in the anonymous repository (Figures 1–2, after and before annealing).

Q4: Please add seeds at 350M/1B.

New results: 350M now has three seeds for every model of Table 1 (Table R6). Table 1's conclusions hold: LIFT is ahead of every baseline in perplexity, RC and arithmetic in all three seeds, with perplexity and RC gaps many times the seed spread, and it is ahead of or level with every baseline on Gen. and MC.

Table R6. 350M, mean ± sd over three seeds.

ModelPPL ↓MC %Gen. %RC BPB ↓Arith. BPB ↓Pattern MC %Pattern BPB ↓
LIFT23.82 ± 0.0144.6 ± 0.423.8 ± 0.70.921 ± 0.0012.13 ± 0.0362.7 ± 1.51.124 ± 0.060
LIFT w/o states24.98 ± 0.0443.9 ± 0.321.8 ± 1.10.944 ± 0.0012.23 ± 0.0161.2 ± 2.21.156 ± 0.037
Transformer, same tokens25.14 ± 0.0243.5 ± 0.521.3 ± 0.90.959 ± 0.0032.31 ± 0.1459.0 ± 0.51.184 ± 0.041
Transformer, same compute24.30 ± 0.0244.5 ± 0.422.7 ± 0.60.938 ± 0.0062.26 ± 0.0660.2 ± 2.21.157 ± 0.046
Distillation24.53 ± 0.0144.2 ± 0.222.4 ± 0.50.936 ± 0.0042.21 ± 0.0760.1 ± 0.81.143 ± 0.035

The 350M seeds alone meant 8 new pretraining runs (two seeds × four models) of 35B+ tokens each, a substantial investment for an academic lab. At 1B, the same would take over 3,000 B200 GPU-hours (over 20K dollars), beyond our academic budget and the rebuttal timeline. One run per size is standard in LLM pretraining: OLMo 2, whose 1B setup we use, also reports one model per size.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of the Revised Paper

We thank the AC and the three reviewers for their time and constructive feedback, which helped improve our paper. We have uploaded a revised PDF that includes all the new experiments and results from the rebuttal. Two reviewers are positive about the work (scores 4 and 3). Reviewer 84tB (score 2) wrote that our response addresses some of their concerns. Their follow-up raised new questions that do not appear in the initial review. We answered them in full, with additional results. Below is a summary of the revisions:

Training past the teacher's budget [Sec. 5.2]

Following Reviewer 84tB's Q1, we trained LIFT and its baselines (3 seeds each) up to 1.45× the teacher's training tokens. LIFT keeps a significant lead over both Transformers and Distillation at every budget. This further supports two claims:

  1. The gain comes from feedback, not from knowledge transfer (84tB W2, QSHN W3). Distillation's gain over the Transformer falls from 0.62 PPL to zero past the teacher's budget, while LIFT stays 1.1–1.2 PPL ahead of Distillation. Once the knowledge signal is gone, Distillation's gain disappears, but LIFT's gain remains: this is the feedback channel's contribution.
  2. LIFT is not bounded by its teacher (84tB W1). At the teacher's own token budget, LIFT beats the teacher in perplexity (28.58 ± 0.05 vs. 29.80) and on every benchmark, in all three seeds. Past that budget, where the teacher is weaker than LIFT, the Transformer still needs a steady 1.7–1.8× LIFT's tokens to match it. LIFT's gains do not depend on a teacher stronger than itself.

Stronger statistical evidence

Reviewers found our evaluation careful (84tB, jDyy). We extend it in two directions:

  1. Seed variance at larger scale [Table 1, App. C.5] (84tB Q4, jDyy W2). We trained three seeds of LIFT and every Table 1 baseline at 350M. LIFT is ahead of every baseline in perplexity, RC and arithmetic in all three seeds. Table 1's 135M and 350M rows are now three-seed means.
  2. Accuracy-based evidence for the procedural claims [Sec. 5.2, App. C.2] (84tB W2, jDyy W1). LIFT has the highest pattern-continuation exact match of the Table 1 models at every scale. Fine-tuned on S5 state tracking at LM scale, LIFT is exact up to N=64, while every Transformer, its teacher included, is near chance from N=32.

Additional comparisons

We ran the additional comparisons the reviewers asked for, and they support the paper's claims:

  1. Distillation from a larger teacher baseline [App. D.4, Table 21] (84tB Q2): with the same 350M teacher, LIFT beats the distilled student in every column.
  2. A parameter-matched control [App. D.5, Table 22] (jDyy Q4): a Transformer widened to LIFT's parameter count (237M → 249M) gains 0.35 PPL, a fifth of LIFT's gain.
  3. Off-the-shelf teachers [App. D.4, Table 20] (QSHN W2): with the public OLMo 2 1B and 7B checkpoints as teachers, LIFT 1B reaches the same perplexity as with our own teacher (18.77 / 18.85 vs. 18.81).
  4. T²MLR trained longer [Sec. 5.3, App. C.7, Table 18] (84tB): continued to 16.2B tokens (3 seeds), T²MLR stays 0.42 PPL behind LIFT at equal tokens, while using 1.6× its training FLOPs.
  5. Hidden states as the state [App. D.6, Table 23] (QSHN Q2): as informative as distributions when fed from the teacher, but suffer from high exposure bias: they lose 3.51 PPL at inference, against 0.74 of LIFT.
  6. What the states carry [App. H, Figures 9–12] (QSHN Q3, jDyy Q5): probing, steering and perturbing the states shows that they carry the intermediate result of the computation, that editing a state can steer the model's answer, and that the pretrained LMs use the whole fed-back distribution, down to its tail.

We thank the reviewers again for their suggestions and efforts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.