acceptodds
Under review as a conference paper at ICLR 2027

INTACT: Inducing Latent Reasoning on a Frozen Backbone

Abstract

Latent reasoning replaces natural-language chains of thought with compact continuous “soft-token” representations, reducing the number of intermediate tokens that must be decoded autoregressively. We use a small external encoder to induce latent reasoning in a frozen backbone model. The encoder iteratively maps the backbone’s hidden states to new latent blocks, constructing a latent reasoning prefix, after which the frozen model writes its remaining reasoning and its answer as text. Our method, INTACT, trains this encoder through distillation to reproduce the effect of written reasoning steps while staying close to the frozen model’s behavior, and then reinforces it on answer correctness. INTACT raises GSM8K accuracy from 75.8% to 79.3% on Llama-3.2-3B and from 77.6% to 79.7% on Qwen-2.5-3B, and cuts end-to-end latency per problem by 19%. Accuracy improves on all four backbones without updating any backbone parameter, and INTACT outperforms the evaluated latent-reasoning baselines at the 3B and 8B scales. A separate encoder trained on a multi-domain corpus improves the same frozen backbone on twelve held-out mathematics, science and commonsense benchmarks without task-specific adaptation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
latent reasoning, frozen backbone, reasoning efficiency, soft tokens

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hjoEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes INTACT, a latent reasoning method built on a frozen backbone language model. It focuses on recovering downstream generation distributions rather than explicitly reconstructing the semantics of reasoning steps. To achieve this, a small external encoder reads the backbone’s hidden states and iteratively generates latent reasoning blocks. The encoder generates latent blocks one at a time using the last hidden states from the backbone. The backbone then generates the remaining textual reasoning and answer. The encoder is first trained through distillation to produce latent prefixes that reproduce the effects of written reasoning steps on subsequent predictions. During this stage, INTACT adopts a preservation loss to maintain the original model’s behavior. Reinforcement learning then refines the encoder using answer correctness as the reward. Experiments on mathematical reasoning tasks across four Llama and Qwen backbones report improved accuracy and higher generation efficiency relative to the frozen LLMs.

Critical strengths
  1. Incorporating latent reasoning into changing backbone parameters is a useful practical setting. Full-parameter tuning on the entire LLM is typically out-of-capacity.
  2. Defining substitution through the subsequent output distribution gives the latent interface a clear functional training target. Personally I found it more meaningful than traditional reasoning-recovery target.
  3. The multi-domain experiment evaluates one Llama-3B encoder checkpoint across twelve benchmarks instead of conducting a specific encoder for each task.
Critical weaknesses
  1. The current method runs a fixed number of reasoning rounds before textual generation. No mechanism for adapting latent depth (when the difficulty is changed) is provided.
  2. Algorithm 2 takes an optimizer step at each depth and regenerates all preceding blocks at the next depth. For a minibatch supporting all K depths, constructing these prefixes requires O(k2)O(k^2) encoder runs. Such design is expensive to extend to deeper latent computation.
Questions for authors

Substitution trains the model to behave as if it had received the replaced reasoning tokens, whereas preservation loss pulls its latent-conditioned distribution toward the model given only the question. How does this regularizer balance beneficial intervention against harmful drift?

2 comments
AuthorsOpenReview ↗

Official Comment to Reviewer hjoE

We thank the reviewer for the thoughtful questions. Below, we address each point with additional experiments. Tables report accuracy (%) / generated tokens unless noted, with bold marking the best result.

W1: INTACT's computation adapts to difficulty through generated text. Although we acknowledge that the number of latent blocks is fixed, the frozen backbone continues reasoning in text, allowing total computation to vary. On GSM8K, grouped by the number of gold reasoning steps, INTACT generates 28% more tokens for the hardest group than for the easiest while improving accuracy over the frozen backbone in every group (Llama-3.2-3B).

Gold steps≤234
Frozen83.4 / 22877.7 / 23872.1 / 277
INTACT84.1 / 16881.0 / 17875.2 / 215

Moreover, in order to test scaling to harder problems, we train INTACT with K=6 instead of K=4 under the same settings. On harder benchmarks, K=6 shows small gains of 0.7–1.8 points with longer generations.

GSM-Symbolic P1GSM-Symbolic P2MATH-500
K=460.6 / 25838.2 / 33746.6 / 511
K=661.3 / 27240.0 / 34647.6 / 553

Finally, we note that on GSM8K, 82% of the problems solved by INTACT can also be solved with only the first two blocks. These findings motivate an adaptive mechanism that learns how many blocks to generate per question, using fewer blocks for easier questions and more for harder ones. We will discuss adaptive latent depth as future work.

W2: Deeper latents add modest training cost. Algorithm 2 requires K(K+1)/2 encoder runs per minibatch. However, the regenerated prefixes are short, so cost grows much more slowly than the run count. Going from K=4 to K=6 doubles the encoder runs from 10 to 21, yet Stage 1 training takes only 34% longer. Prefix regeneration is required only during training, whereas at inference each block is computed once.

We would also like to thank the reviewer for the suggestion in the first point, as the same argument applies here. As shown in W1, 82% of the problems INTACT solves can be solved with only the first two blocks, so an adaptive mechanism that generates fewer blocks for easier questions would also reduce the number of encoder runs.

Q1: Section 4.3 compares preservation anchors on Qwen2.5-3B:

Preservation anchorGSM8KOODTokens
Frozen backbone77.674.6273
None (β=0)42.954.580
Gold chain74.069.1240
Frozen rollout76.276.2309
Student rollout (INTACT)78.875.5246

Without preservation, outputs shorten to 80 tokens, while GSM8K accuracy falls 34.7 points below the frozen backbone. The substitution term is evaluated on the reference trace, whereas preservation uses the student's own latent-conditioned rollout and encourages it to remain plausible under the frozen model. The gold-chain and frozen-rollout anchors evaluate traces not produced by the student, so they may miss errors that arise during inference. Only the student-rollout anchor improves both GSM8K and OOD accuracy over the frozen backbone while generating fewer tokens.

I appreciate the authors' response, which addresses most of my concerns. I'll maintain my score.

Reviewer_qw4POpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes INTACT, which uses an additional encoder to generate a latent prefix for a frozen backbone. Unlike traditional latent reasoning models such as Coconut and CODI, which achieve latent reasoning through the backbone’s own recurrent hidden states, INTACT relies on an external encoder.

Critical strengths
  1. With the additional encoder, INTACT does not modify the original model, so its text-based slow thinking is preserved.
  2. The design of INTACT is compelling and can be utilised to perform hybrid reasoning.
  3. The experiments are thorough, and the results are satisfactory in terms of both accuracy and efficiency.
Critical weaknesses
  1. There appear to be some issues with the comparison: the accuracy of the baseline model is affected by truncation.
  2. The reduction in tokens appears to be more a side effect of RL than a result of “replacing the first four steps”. Normalisation by sequence length favours shorter correct answers, which is a known finding from Dr. GRPO.
  3. The datasets used in this paper are relatively simple. It is recommended to use slightly more complex benchmarks such as MATH-500, AIME and AMC. For out-of-domain tasks, datasets such as GPQA can be utilised.
  4. The permutation controls in the appendix are reported only for Stage 1. For the Stage 2 encoder, which produces the headline results, it cannot be guaranteed whether the latent prefix serves as a key element of the reasoning or merely as a general reasoning prefix.
Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Official Comment to Reviewer qw4P

We thank the reviewer for the suggestions. Below, we address each point with additional experiments. Tables report accuracy (%) / generated tokens, with bold and underlining marking the best and second-best results.

W1: Truncation does not affect the frozen baselines. To test whether the results are affected by truncation, we increased the generation limit from 768 to 2,048 tokens for the frozen backbones.

Frozen, GSM8K7682,048
Llama-3.2-1B42.642.7
Llama-3.2-3B75.675.7
Llama-3.1-8B73.073.2

On the smaller backbones, accuracy moves by only tenths of a point, whereas frozen Llama-3.1-8B rarely terminates. Even at 10,240 tokens, 95.8% of its generations remain unfinished, and its accuracy stays at 73.1%. INTACT instead achieves 83.6% accuracy in 182 tokens with the same frozen weights and prompt. Hence, INTACT's gains are not a truncation artifact.

W2: Stage 1 provides most of the shortening. We agree that Stage 2's length normalization could have contributed to shortening. However, INTACT's primary shortening mechanism is to replace token-based reasoning steps with latents trained to reproduce their effect. To assess whether Stage 2 alone suffices, we omit Stage 1 and train the encoder directly with Stage 2, the GRPO procedure in the paper, on Llama-3.2-3B:

GSM8KOOD
Frozen75.8 / 26166.5 / 224
Stage 2 only38.8 / 51033.0 / 506
INTACT79.3 / 19568.8 / 165

Without Stage 1, accuracy collapses, generations lengthen, and half fail to terminate. These results support step-wise replacement as the basis for stable shortening.

W3: INTACT also improves accuracy on harder benchmarks. We provide results on harder benchmarks below. All systems are trained on GSM8K-Aug. For LOTUS, we use the authors' released Llama-3.2-3B checkpoint, evaluated with their script. LOTUS counts include its 150 latents. INTACT is the most accurate system in every comparison and improves on the frozen backbone with fewer tokens, achieving the highest AIME pass@3. In contrast, Aug-SFT and LOTUS, which fine-tune the backbone on the same data, fall far below the frozen backbone on MATH-500, AIME, and AMC. We believe that this is because both Aug-SFT and LOTUS trained on GSM8K prompts change the backbone's reasoning capabilities significantly.

MATH-500

BackboneFrozenAug-SFTLOTUSINTACT
Llama-3B4̲4̲.̲0̲ / 78412.0 / 637.6 / 1̲5̲4̲46.6 / 511
Llama-8B4̲6̲.̲4̲ / 1,99412.6 / 82–50.4 / 4̲9̲0̲

GPQA

BackboneFrozenAug-SFTLOTUSINTACT
Llama-8B20.2 / 1,8862̲3̲.̲2̲ / 45–25.3 / 7̲1̲0̲

AIME (pass@3)

BackboneFrozenAug-SFTLOTUSINTACT
Llama-3B1̲3̲.̲3̲ / 1,4550.0 / 1090.0 / 1̲5̲3̲17.8 / 1,008

AMC (pass@3)

BackboneFrozenAug-SFTLOTUSINTACT
Llama-3B3̲3̲.̲7̲ / 1,3429.6 / 876.0 / 1̲5̲3̲38.6 / 837

GSM-Symbolic (Mirzadeh et al., 2024). P1 and P2 add one and two phrases, respectively, to increase problem complexity.

Llama-3BMainP1P2
Frozen6̲6̲.̲3̲ / 2735̲5̲.̲5̲ / 3313̲7̲.̲6̲ / 426
Aug-SFT61.6 / 3545.2 / 4721.8 / 57
LOTUS55.5 / 1̲5̲3̲30.0 / 1̲5̲4̲9.4 / 1̲5̲6̲
INTACT69.3 / 20260.6 / 25838.2 / 337

W4: Stage 2 preserves question- and order-specific conditioning. We perform the permutation controls on the Stage 2 encoder (Llama-3.1-8B).

Stage 2GSM8KOOD
INTACT83.2 / 17171.9 / 143
Block shuffle81.0 / 20370.6 / 176
Other question's prefix77.3 / 25567.7 / 259

Shuffling latent blocks or substituting another question's prefix reduces accuracy and lengthens generation, supporting order- and question-specific conditioning after Stage 2.

Furthermore, to isolate question-specific conditioning, we replace the encoder outputs with a fixed prefix of 16 learned vectors shared across questions, trained with the same multi-domain settings (Llama-Nemotron, Llama-3.2-3B). Columns macro-average four math, five science, and three commonsense benchmarks.

Macro accuracy (%)MathScienceCommonsense
Frozen68.857.442.6
Fixed prefix, Stage 270.964.648.5
INTACT, Stage 271.173.661.6

INTACT exceeds the fixed prefix by 9.0 points on science and 13.1 on commonsense, further showing that its gains depend on question-specific conditioning. We will add these results to the appendix.

Reviewer_BK7BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a method for training an efficient latent chain of thought on top of a frozen language model, using an additional lighter-weight model. The paper demonstrates that the training improved over the base frozen model. However, this could be merely due to receiving more training signal from the new data. My biggest concern is that the argument for having a frozen back-bone, as given by the authors, is that it will avoid catastrophic forgetting. However, some of their baselines that update model weights, such as Nemo-SFT, improve both in Math and OOD settings beyond the INTACT's gains -- suggesting the backbone model training did not cause degradation in performance. I also think to establish teh usefulness of INTACT, it should also be compared with a baseline that retains the base model by just training a lora with reward for concise reasoning on the same amount of data. The remaining benefit would have been if the cost of generation was lesser; but that is also not very clear in figure 3 ... it seems that INTACT is right on the curves of trade-off .... but not drastically better in compute with same performance gains. Another question I have is that the efficiency is not reported in FLOPs when comparing to all other models. token count can be particularly misleading when we are repeatedly switching between backbone model and latent generation model, the actual FLOPs will depend on how much of the KV-Cache was re-used.

Critical strengths

The paper presents a method for training an efficient latent chain of thought on top of a frozen language model, using an additional lighter-weight model. The paper demonstrates that the training improved over the base frozen model. However, this could be merely due to receiving more training signal from the new data.

Critical weaknesses
  • My biggest concern is that the argument for having a frozen back-bone, as given by the authors, is that it will avoid catastrophic forgetting. However, some of their baselines that update model weights, such as Nemo-SFT, improve both in Math and OOD settings beyond the INTACT's gains -- suggesting the backbone model training did not cause degradation in performance.
  • I also think to establish the usefulness of INTACT, it should also be compared with a baseline that retains the base model by just training a lora with reward for concise reasoning on the same amount of data.
  • The remaining benefit would have been if the cost of generation was lesser; but that is also not very clear in figure 3 ... it seems that INTACT is right on the curves of trade-off .... but not drastically better in compute with same performance gains. Another question I have is that the efficiency is not reported in FLOPs when comparing to all other models. token count can be particularly misleading when we are repeatedly switching between backbone model and latent generation model, the actual FLOPs will depend on how much of the KV-Cache was re-used.
  • I also think that the comparison with SoftCoT should be more thorough across multiple models, since in terms of setup it is closest to INTACT and the comparison given in appendix shows small gain.
Questions for authors

I think to establish the usefulness of INTACT, it should also be compared with a baseline that retains the base model by just training a lora with reward for concise reasoning on the same amount of data. The efficiency is not reported in FLOPs when comparing to all other models. token count can be particularly misleading when we are repeatedly switching between backbone model and latent generation model, the actual FLOPs will depend on how much of the KV-Cache was re-used.

2 comments
AuthorsOpenReview ↗

Official Comment to Reviewer BK7B

We thank the reviewer for the suggestions. Below, we address each point with additional experiments. Tables report accuracy (%) / generated tokens unless noted.

W1: We first clarify that our forgetting argument concerns latent-reasoning methods that update the backbone on a narrow domain. Nemo-SFT is not a latent-reasoning method and is trained on the Llama-Nemotron mixture, a diverse corpus, so we do not expect it to forget. However, when comparing INTACT with Aug-SFT and LOTUS, none of which see MATH-500, AMC, or SVAMP in training, INTACT is the only approach that improves over the frozen backbone while reducing tokens.

Llama-3.2-3BMATH-500AMCSVAMP
Frozen44.0 / 78433.7 / 1,34277.0 / 200
Aug-SFT12.0 / 639.6 / 8774.5 / 19
LOTUS7.6 / 1546.0 / 15374.0 / 153
INTACT46.6 / 51138.6 / 83781.8 / 139

As Nemo-SFT does not suffer from forgetting, we instead compare the two on efficiency and generalization beyond the training corpus. Mathematics and science are part of Nemo-SFT's training corpus, but commonsense (ECQA, SocialIQA, HellaSwag), reading comprehension (RACE-high), logical reasoning (LogiQA, FOLIO), and general knowledge (MMLU) are not.

Llama-3.2-3BFrozenNemo-SFTINTACT
ECQA52.6 / 21467.0 / 13064.2 / 61
SocialIQA42.9 / 17746.7 / 14164.3 / 143
HellaSwag32.4 / 77652.5 / 13856.2 / 60
RACE-high56.1 / 38063.9 / 14768.0 / 143
LogiQA25.2 / 61736.9 / 23237.3 / 171
FOLIO38.4 / 12043.8 / 13345.3 / 108
MMLU48.3 / 40660.5 / 17860.2 / 160

INTACT outperforms Nemo-SFT on five of seven benchmarks, raising mean commonsense accuracy from 55.4% to 61.6%, and uses fewer tokens on six. Therefore, INTACT is more efficient and generalizes better to domains unseen by Nemo-SFT.

W2: INTACT outperforms a LoRA trained for concise reasoning. We trained a LoRA adapter on Llama-3.2-3B with a reward for concise reasoning, the length-penalized reward of Arora and Zanette (2025), following their setup with α=0.1\alpha = 0.1 and an RLOO advantage. Both approaches are trained in the same setting and evaluated at matched steps, with LoRA averaged over two seeds.

Llama-3.2-3BGSM8KOOD
Frozen75.8 / 26166.5 / 224
LoRA70.1 / 7965.0 / 65
INTACT79.3 / 19568.8 / 165

LoRA shortens generation substantially, but falls 5.7 points below the frozen backbone on GSM8K. INTACT improves on the frozen backbone while also shortening generation. Stage 1 trains each latent to reproduce the effect of the reasoning step it replaces, so the shortening comes from replacing steps rather than omitting them. The LoRA policy instead compresses the reasoning itself, which costs accuracy.

W3: INTACT reduces FLOPs while improving accuracy. We compute FLOPs per question following Kaplan et al. (2020), with 2N2N FLOPs per position, where NN is the number of backbone parameters used in matrix multiplications. INTACT reuses the backbone's KV cache throughout. Each latent block is appended to the cache rather than re-encoding the question, so alternating with the encoder adds only 16 positions.

We convert every system in Figure 3 to FLOPs (Llama-3.2-3B, GSM8K).

Acc.TFLOPs
Frozen75.82.06
Aug-SFT71.50.60
CODI60.80.47
MUX65.00.47
LOTUS70.07.17
INTACT79.31.75

Cheaper methods fall 4 to 15 points below the frozen backbone. LOTUS passes its 150 latents through the backbone seven times, costing about 4× INTACT. Across all four backbones, INTACT is also cheaper than the frozen backbone, saving 5% to 73% of FLOPs.

TFLOPsFrozen GSM8KINTACT GSM8KFrozen OODINTACT OOD
Llama-1B0.770.730.640.59
Llama-3B2.061.751.741.48
Llama-8B12.293.8811.963.28
Qwen-3B2.071.391.561.16

W4: INTACT outperforms SoftCoT on a second backbone. Appendix A uses Llama-3.1-8B, one of SoftCoT's original backbones. We extend the comparison to Llama-3.2-3B, training SoftCoT with the authors' code and recipe. The backbone and the Llama-3.2-1B assistant are frozen and only the projection is trained. Both methods use the SoftCoT protocol.

Llama-3.2-3BGSM8KSVAMPGSM-HardMultiArithOOD
SoftCoT73.679.523.796.866.7
INTACT75.680.325.697.367.7

INTACT is more accurate on all four benchmarks, by 2.0 points on GSM8K and 1.0 out of domain. Together with the 8B comparison, INTACT leads on 9 of the 10 accuracy entries, without running a second language model at inference.

Reviewer_BK7BOpenReview ↗

response addresses concerns

The authors' response address all my concerns to some level, I will adjust my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.