- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes INTACT, a latent reasoning method built on a frozen backbone language model. It focuses on recovering downstream generation distributions rather than explicitly reconstructing the semantics of reasoning steps. To achieve this, a small external encoder reads the backbone’s hidden states and iteratively generates latent reasoning blocks. The encoder generates latent blocks one at a time using the last hidden states from the backbone. The backbone then generates the remaining textual reasoning and answer. The encoder is first trained through distillation to produce latent prefixes that reproduce the effects of written reasoning steps on subsequent predictions. During this stage, INTACT adopts a preservation loss to maintain the original model’s behavior. Reinforcement learning then refines the encoder using answer correctness as the reward. Experiments on mathematical reasoning tasks across four Llama and Qwen backbones report improved accuracy and higher generation efficiency relative to the frozen LLMs.
Critical strengths
- Incorporating latent reasoning into changing backbone parameters is a useful practical setting. Full-parameter tuning on the entire LLM is typically out-of-capacity.
- Defining substitution through the subsequent output distribution gives the latent interface a clear functional training target. Personally I found it more meaningful than traditional reasoning-recovery target.
- The multi-domain experiment evaluates one Llama-3B encoder checkpoint across twelve benchmarks instead of conducting a specific encoder for each task.
Critical weaknesses
- The current method runs a fixed number of reasoning rounds before textual generation. No mechanism for adapting latent depth (when the difficulty is changed) is provided.
- Algorithm 2 takes an optimizer step at each depth and regenerates all preceding blocks at the next depth. For a minibatch supporting all K depths, constructing these prefixes requires encoder runs. Such design is expensive to extend to deeper latent computation.
Questions for authors
Substitution trains the model to behave as if it had received the replaced reasoning tokens, whereas preservation loss pulls its latent-conditioned distribution toward the model given only the question. How does this regularizer balance beneficial intervention against harmful drift?
2 comments
I appreciate the authors' response, which addresses most of my concerns. I'll maintain my score.
Official Comment to Reviewer hjoE
We thank the reviewer for the thoughtful questions. Below, we address each point with additional experiments. Tables report accuracy (%) / generated tokens unless noted, with bold marking the best result.
W1: INTACT's computation adapts to difficulty through generated text. Although we acknowledge that the number of latent blocks is fixed, the frozen backbone continues reasoning in text, allowing total computation to vary. On GSM8K, grouped by the number of gold reasoning steps, INTACT generates 28% more tokens for the hardest group than for the easiest while improving accuracy over the frozen backbone in every group (Llama-3.2-3B).
Moreover, in order to test scaling to harder problems, we train INTACT with K=6 instead of K=4 under the same settings. On harder benchmarks, K=6 shows small gains of 0.7–1.8 points with longer generations.
Finally, we note that on GSM8K, 82% of the problems solved by INTACT can also be solved with only the first two blocks. These findings motivate an adaptive mechanism that learns how many blocks to generate per question, using fewer blocks for easier questions and more for harder ones. We will discuss adaptive latent depth as future work.
W2: Deeper latents add modest training cost. Algorithm 2 requires K(K+1)/2 encoder runs per minibatch. However, the regenerated prefixes are short, so cost grows much more slowly than the run count. Going from K=4 to K=6 doubles the encoder runs from 10 to 21, yet Stage 1 training takes only 34% longer. Prefix regeneration is required only during training, whereas at inference each block is computed once.
We would also like to thank the reviewer for the suggestion in the first point, as the same argument applies here. As shown in W1, 82% of the problems INTACT solves can be solved with only the first two blocks, so an adaptive mechanism that generates fewer blocks for easier questions would also reduce the number of encoder runs.
Q1: Section 4.3 compares preservation anchors on Qwen2.5-3B:
Without preservation, outputs shorten to 80 tokens, while GSM8K accuracy falls 34.7 points below the frozen backbone. The substitution term is evaluated on the reference trace, whereas preservation uses the student's own latent-conditioned rollout and encourages it to remain plausible under the frozen model. The gold-chain and frozen-rollout anchors evaluate traces not produced by the student, so they may miss errors that arise during inference. Only the student-rollout anchor improves both GSM8K and OOD accuracy over the frozen backbone while generating fewer tokens.