- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes LIFT, a Transformer that feeds a "state" — its own top-k next-token distribution — back as an additional input at the next position, giving a deep-to-shallow channel beyond the sampled token. A small fusion layer (~3% of parameters at 1B) merges the embedded state with the token embedding. To keep pretraining parallel, input states are teacher-forced from a frozen pretrained LM run over the same text, and the student is trained with next-token loss plus a top-k forward-KL alignment to the teacher; in the last 10% of training it adapts to its own states, and at inference it feeds back its own predictions. On S5 permutation composition, a 2-layer LIFT solves lengths where same-depth Transformers fail, even with a teacher that itself fails. Pretraining OLMo 2-style models at 135M/350M/1B for 5× Chinchilla tokens, LIFT outperforms token-matched Transformers on perplexity and most downstream metrics, is level with or ahead of compute-matched Transformers (counting the teacher's forward pass), beats a distillation baseline, and compares favorably with Jacobi-iteration feedback models (T2MLR and a reimplementation of the Full-bandwidth Transformer) at matched tokens and compute.
Critical strengths
-
A simple, scalable way to pretrain a latent feedback channel. Teacher-forcing the fed-back states from an existing pretrained LM keeps training fully parallel, avoiding the approximation-quality vs. training-cost tradeoff of the Jacobi-iteration approaches (T2MLR, Full-bandwidth/Multi-pass Transformer), and is a different route from those works. It works with a same-size teacher (larger teachers do not consistently help; Tables 7–9, 11), and on S5 even a teacher that fails the task suffices, so the requirement on the teacher is mild. Since capable open pretrained models are readily available, treating the teacher as an existing resource is reasonable.
-
Careful, well-controlled experiments. The paper compares against token- and compute-matched Transformers (with the teacher's forward pass charged), a distillation baseline that isolates the alignment objective (plus tuned distillation variants, App. D.1), the same model run without states, and reproduced feedback baselines, and includes informative ablations (teacher as input vs. target vs. none, channel width, trunk vs. adaptation phase). The S5 experiment cleanly illustrates the expressivity motivation.
Critical weaknesses
1. The reasoning claims rest mainly on likelihood metrics. The abstract claims gains on "downstream reasoning tasks" and "procedural tasks", but at these scales the evidence is largely in bits per byte rather than task accuracy. On the authors' own arithmetic task, exact-match accuracy is roughly 3–6% for every model from three operands on (Table 12), so the reported arithmetic gains are BPB differences on answers no model gets right; pattern continuation is likewise headlined in BPB. Eight of the twenty downstream suites, including GSM8K, MATH, HumanEval and MBPP, are at chance or floor for all models and are excluded from the main table (§5.1, App. B.4). The likelihood improvements are real, but they do not establish improved multi-step reasoning; accuracy-based evidence (e.g., at larger scale, or on tasks where models are above floor) is needed to support those claims.
2. Statistical strength of the matched-compute comparison. Against the compute-matched Transformer the margins are small (e.g., at 1B: PPL 18.81 vs. 19.17, MC 52.5 vs. 51.6, Gen. 35.0 vs. 34.8), and the 350M and 1B models are trained with a single seed; the bootstrap intervals in Table 6 capture evaluation-set variance, not training variance. At 135M, where three seeds are available, most downstream differences at matched compute are within seed noise (App. D.1), and in the §5.3 setting LIFT is level with the compute-matched Transformer on perplexity and slightly behind on MC and RC (Table 14). The token-matched gains are clear; the compute-matched downstream gains would be more convincing with seed variance at larger scale.
Questions for authors
- Can you report accuracy-based results that support the reasoning claims, e.g., exact-match arithmetic or pattern continuation at larger scale, or tasks where the models are above floor? Do the BPB gains translate into accuracy anywhere?
- Can you report seed variance for the 350M and/or 1B compute-matched comparison (even two additional seeds), so that the downstream margins can be assessed?
- Since teacher-forced states and Jacobi-style refinement address the parallel-training problem in different ways, have you considered combining them (e.g., using teacher states as the initialization for refinement passes, or refining after teacher-forced pretraining)? Would they be complementary?
- The Transformer baselines are LIFT without the fusion layer, so LIFT has more parameters (~3% at 1B; 138.2M vs. 134.5M in §5.3), and the compute-matched baseline receives extra tokens rather than parameters. The ablations with LIFT's exact parameter count (no-teacher, teacher-as-input-only, k = 1, LIFT w/o states) suggest the extra parameters are not the source of the gain, but a direct parameter-matched control (e.g., a vanilla Transformer widened or deepened to LIFT's parameter count, at matched tokens and compute) would make this explicit. Could you include it?
- Does teacher supervision cause the model to use the recurrent path less, i.e., to consume the teacher's (and later its own) states as an informative input rather than learning to carry its own computation through the channel? Could you analyze how the channel is used in the pretrained models, e.g., how much information the fed-back states carry beyond the teacher's next-token distribution, how performance changes when states are perturbed or replaced, or how a small-scale LIFT trained with exact recurrence compares in its reliance on the channel?
3 comments
Response to Reviewer jDyy (2/3)
Response to Reviewer jDyy (2/3)
W2 + Q2: Seed variance at larger scale
New: we trained two more seeds at 350M, which now has three seeds for every model of Table 1.
Table R6. 350M, mean ± sd over three seeds.
| Model | PPL ↓ | MC % | Gen. % | RC BPB ↓ | Arith. BPB ↓ |
|---|---|---|---|---|---|
| LIFT | 23.82 ± 0.01 | 44.6 ± 0.4 | 23.8 ± 0.7 | 0.921 ± 0.001 | 2.13 ± 0.03 |
| LIFT w/o states | 24.98 ± 0.04 | 43.9 ± 0.3 | 21.8 ± 1.1 | 0.944 ± 0.001 | 2.23 ± 0.01 |
| Transformer, same tokens | 25.14 ± 0.02 | 43.5 ± 0.5 | 21.3 ± 0.9 | 0.959 ± 0.003 | 2.31 ± 0.14 |
| Transformer, same compute | 24.30 ± 0.02 | 44.5 ± 0.4 | 22.7 ± 0.6 | 0.938 ± 0.006 | 2.26 ± 0.06 |
| Distillation | 24.53 ± 0.01 | 44.2 ± 0.2 | 22.4 ± 0.5 | 0.936 ± 0.004 | 2.21 ± 0.07 |
The three seeds at 135M (the paper's Table 10) and 350M (Table R6), with the 1B run (the paper's Table 1), give 7 independently trained pairs of LIFT and the compute-matched Transformer. LIFT is ahead:
- on perplexity, arithmetic, Gen. and RC in all 7 pairs (Table 9 in the anonymous repository);
- on MC in only 4 of 7 pairs, so at matched compute MC is level (44.6 ± 0.4 vs 44.5 ± 0.4 at 350M), and the revised §5.2 will say so;
- on 61 of 84 individual suite comparisons. Table 1's MC, Gen. and RC columns are means over 12 suites (App. B.4). Comparing the two models on each suite separately, in each of the 7 pairs, gives 84 comparisons; LIFT is ahead in 15 of 21 on the MC suites, 12 of 21 on the generative suites and 34 of 42 on the BPB suites.
The perplexity lead is many times its seed spread: 0.47 ± 0.08 at 135M, 0.50 / 0.48 / 0.45 in the three 350M seeds, and 0.36 at 1B. In tokens, the Transformer needs 1.6× / 2.1× / 2.0× LIFT's tokens to reach its loss at 135M / 350M / 1B, while LIFT's training compute per token is 1.42–1.44× a Transformer's.
Q1. Gains in terms of accuracy
Yes, on pattern continuation, where BPB and accuracy are measured on the same questions (W1, point 1). LIFT's BPB lead comes with the highest generative EM (Table R2a) and the highest MC accuracy (Table 11 in the anonymous repository; tied at 135M) of the Table 1 models at every scale.
Q3. Combining teacher-forced states with Jacobi-style refinement
We believe these methods are complementary, and LIFT already combines them in a simple form: its adaptation phase (App. A.3) drops the teacher and trains on the model's own states, produced by one or two parallel passes, i.e., refinement after teacher-forced pretraining. The ablations show the two add up (App. D.1, D.3): refinement alone, without a teacher, only ties the compute-matched Transformer at 135M; at 1B the teacher-forced trunk alone keeps 73% of LIFT's gain, and the refinement phase adds the rest. The fuller combination the reviewer suggests, the teacher as a bootstrap followed by multi-pass Jacobi refinement, is a direction we are actively exploring.
Response to Reviewer jDyy (3/3)
Response to Reviewer jDyy (3/3)
Q4. A direct parameter-matched control
We conducted the experiment the reviewer suggested. At 135M the fusion layer is a larger share of LIFT's parameters (4.6%, against 3% at 1B), so this is where extra parameters could matter most. We widened the Transformer's MLP (ratio 9 instead of 8) to 249.6M parameters, against LIFT's 248.5M; its added FLOPs per token match LIFT's (the fusion layer plus the k = 1024 state embedding). We trained it with Table 1's seed and data order at matched tokens (13.4B) and at matched compute (Table R8; also Table 14 in the anonymous repository).
Table R8. 135M, seed of Table 1.
| Model | Params | Tokens | PPL ↓ | Arith. BPB ↓ | MC % | Gen. % | RC BPB ↓ |
|---|---|---|---|---|---|---|---|
| Transformer | 237.0M | 13.4B | 32.43 | 2.46 | 38.1 | 13.6 | 1.110 |
| Transformer, parameter-matched | 249.6M | 13.4B | 32.08 | 2.47 | 38.5 | 13.3 | 1.106 |
| Transformer, parameter-matched | 249.6M | 17.9B | 30.99 | 2.46 | 39.5 | 12.5 | 1.079 |
| LIFT | 248.5M | 13.4B | 30.65 | 2.29 | 39.4 | 15.5 | 1.078 |
At matched tokens, the extra parameters give the Transformer 0.35 PPL, a fifth of LIFT's gain (1.78). At matched compute, LIFT is 0.34 PPL ahead of the parameter-matched Transformer and clearly ahead on arithmetic and Gen.; MC and RC are level. Together with the exact-parameter ablations the reviewer notes, this shows that the gain does not come from the added parameters.
Q5. Teacher supervision and recurrent path utilization
It is challenging to separate “informative input” from “carrying its own computation” (the model's own computation is itself an informative input), but here are results from our analyses of these questions:
- Exact recurrence vs. state teacher forcing (the S5 models of Sec. 4). LIFT-RNN is trained with full backpropagation through the recurrence, so it has an explicit signal to put relevant computation on the state. Its teacher-forced students never receive this signal in training, yet they inherit this property from the RNN teacher and extrapolate beyond the training lengths.
- Perturbing states in the pretrained LMs (135M / 1B). Switching the channel off costs 1.8 / 0.8 PPL, and another sequence's states are far worse than none (52.3 vs 31.6 PPL at 135M). The top 256 tokens keep 89% of the gain, while a one-hot of the top token (an input never seen in training) is worse than no state (Figure 5 in the anonymous repository plots the C4 perplexity of every state variant against switching the channel off; Table 16 lists the values).
We thank the reviewer again, and hope our response resolves their concerns.
Response to Reviewer jDyy (1/3)
Response to Reviewer jDyy (1/3)
We thank Reviewer jDyy for the careful review. We are glad the reviewer sees LIFT as "a simple, scalable way to pretrain a latent feedback channel", finds treating the teacher as an existing resource reasonable and the experiments careful and well controlled. We ran new experiments to address the reviewer’s questions. All new results and analyses below will be in the revised paper PDF. The full tables and figures are already available in an anonymous repository: https://anonymous.4open.science/r/lift-rebuttal-results-0367/.
W1: The reasoning claims rest mainly on likelihood metrics
We agree that accuracy is the stronger evidence. At these scales, however, exact match is at the floor on most reasoning tasks for every model. We therefore rely on the common practice of using BPB as a proxy for performance in small-scale experiments: OLMo3's evaluation suite uses BPB tasks on math, code and QA for its small-scale experiments, as our RC column does, DataDecide (Magnusson et al., ICML 2025) finds that likelihood metrics at small scale predict rankings at larger scale better than accuracy, and Heineman et al. (NeurIPS 2025) find that BPB has a far higher signal-to-noise ratio than accuracy on GSM8K and MBPP, two of the suites at floor here.
We will revise our reasoning claims accordingly: at matched compute, LIFT's advantage will be stated as a likelihood result. Still, we add accuracy benchmarks to the main text to better support the reasoning claim:
1. Pattern continuation, with a new accuracy metric: This task (OLMES, 534 questions) asks the model to continue a sequence by a rule shown in context (e.g., 2 4 6 8 → 10; @ # % @ # → %). Besides the multiple-choice (MC) accuracy already in App. C.6, we measured the exact-match (EM) accuracy of the generated continuation. LIFT has the highest EM of the Table 1 models at every scale, +4.5 / +5.9 / +5.3 points over the token-matched Transformer at 135M / 350M / 1B (Table R2a), consistent with the best BPB results we report in the main text. MC accuracy agrees: LIFT is the highest of the Table 1 models at every scale (tied at 135M) and ahead of the token-matched Transformer in all 7 independently trained pairs (3 seeds for 135M and 350M, 1 seed for 1B) on both metrics. At 1B it also beats its teacher, trained on 2× the tokens (Tables 10–11 in the anonymous repository). Following the reviewer's concern, we will move these accuracy results (existing and new) from the appendix (the paper's Table 13) to the main text, with three seeds at 135M and 350M.
Table R2a. Pattern continuation, generative exact match (%). 135M and 350M: mean ± sd over three seeds, 1B and the teacher: one run.
2. New benchmark: S5 state tracking at LM scale. We fine-tuned all the pretrained models in Table 1 on the S5 task of Sec. 4 (8,000 steps on 1–16 permutations, loss on the state after every step). LIFT is fine-tuned with the paper's training recipe: its input states come from a Transformer baseline fine-tuned the same way, with the alignment loss. LIFT is exact at every length up to N=64, although its teacher falls to chance from N=32, (Table R3). The results are the same at 135M and 350M (Table 13 in the anonymous repository).
Table R3. S5 accuracy (%) of the fine-tuned 1B models on sequences of length N; chance is 0.8%.