acceptodds
Under review as a conference paper at ICLR 2027

ThinkBridge: Eliciting Latent Reasoning from Large Reasoning Models

Abstract

Explicit chain-of-thought (CoT) helps large language models (LLMs) solve problems, but generating long CoTs token by token incurs substantial inference latency. Existing latent reasoning methods fine-tune LLMs to carry intermediate computation in continuous representations. Reasoning-focused post-training has produced large reasoning models (LRMs), a class of LLMs with strong explicit reasoning capabilities. In preliminary mathematical reasoning experiments, we examine conventional supervised fine-tuning and existing latent reasoning methods on LRMs and find marked accuracy drops in some settings relative to the unmodified models. We also find that the original LRMs provide complementary successful responses in thinking and non-thinking modes. These observations motivate ThinkBridge, which keeps the LRM frozen and trains a lightweight reasoner using the model's own successful responses. The reasoner constructs compact latent states that guide the LRM to respond without first generating an explicit CoT. Training combines three objectives: supervision on correct native responses supports accurate answers; on-policy distillation aligns latent-conditioned and full-CoT-conditioned response predictions, training latent states to guide responses in place of CoT; and contrastive learning encourages question-specific latent states that support reasoning about the current question. We train ThinkBridge on GSM8K-Aug-NL. Across five mathematical benchmarks, it achieves the highest mean accuracy among the evaluated latent reasoning methods at 0.6B and 4B, with mean times to first response token (TTFT) of 45.50 and 56.33 ms, respectively. It also outperforms these baselines in multi-turn accuracy without dialogue-specific training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Latent Reasoning, Large Reasoning Models, On-policy Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_CfrgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ThinkBridge, which keeps a large reasoning model (Qwen3) frozen and trains a lightweight external reasoner (about 5% of the backbone’s parameters) to map the question’s hidden states into a short sequence of latent states that guide the model to answer without generating an explicit thinking trace. The reasoner is trained on the model’s own correct responses from both thinking and non-thinking modes, using cross-entropy supervision, on-policy distillation that aligns latent-conditioned predictions with full-CoT-conditioned ones, and a contrastive objective that encourages question-specific latent states. On five math benchmarks, ThinkBridge achieves the highest mean accuracy among the evaluated latent reasoning methods at 0.6B and 4B, with a time to first response token of about 45–56 ms, and it also leads these baselines on a three-turn math dialogue task without dialogue-specific training.

Critical strengths
  1. Lightweight and cheap at inference. The reasoner adds only about 5% of the backbone’s parameters (29.4M for 0.6B, 183.7M for 4B, Table A1), and TTFT is 45–56 ms, far below CoT-SFT (1.4–2.0 s) and all latent baselines (218–606 ms), and stable across benchmarks (Table G1).
  2. Generalizes to multi-turn dialogue without dialogue training. Trained only on single-turn questions, the same reasoner leads all latent baselines at every turn on MathChat; at 4B, turn-two and turn-three accuracies (69.34% and 61.77%) stay close to non-thinking mode (70.80% and 62.77%), while the latent baselines collapse to 7–21%.
Critical weaknesses
  1. No gain over the untrained non-thinking mode, which is the baseline that actually matters. At 0.6B, ThinkBridge averages 63.53% versus 63.97% for non-thinking mode, while being twice as slow (45.50 vs. 20.74 ms TTFT), and it is 6.3 points worse on SVAMP. At 4B, the gain is only 1.09 points from a single seed. On MATH-500, the two are essentially identical (50.40% vs. 50.40%; 82.40% vs. 82.00%). The method adds a trained module and extra latency for no clear benefit.
  2. No evidence that the latent states carry reasoning. There is no test replacing zz with zeros, random vectors, or another question’s latent states, and no comparison with a question-independent learned prefix (standard prompt tuning). Figure F1 shows ThinkBridge and non-thinking mode agree on most questions (only 5.9% vs. 4.2% uniquely solved at 4B). The simplest explanation is that zz acts as a soft prompt nudging the response style, not as latent reasoning.
  3. TTFT hides the real latency. TTFT stops at the first response token, so reasoning written inside the response is never counted, while CoT-SFT’s reasoning is. This makes ThinkBridge look 30× faster than CoT-SFT even if both produce similar amounts of reasoning text. Total generation time or total output tokens is needed.
Questions for authors
  1. At 0.6B, ThinkBridge averages 63.53% in Table 1, below the untrained non-thinking mode (63.97%), while also having a higher TTFT (45.50 vs. 20.74 ms); on SVAMP the gap is 6.3 points (76.33% vs. 82.67%). At 4B, the gain over non-thinking mode is 1.09 points from a single seed. What practical advantage does ThinkBridge offer over simply using the backbone’s non-thinking mode? Could you report results over multiple seeds to show whether the 4B gain is statistically meaningful?
  2. On MATH-500, ThinkBridge matches non-thinking mode almost exactly (50.40% vs. 50.40% at 0.6B; 82.40% vs. 82.00% at 4B), even though the reasoner is trained only on GSM8K-Aug-NL. How do you rule out the explanation that the reasoner barely changes the model’s behavior out of domain? Could you report how often ThinkBridge’s MATH-500 responses differ from non-thinking responses, in final answers or in the text itself?
  3. Figure F1 shows that at 4B, ThinkBridge and non-thinking mode jointly solve 72.8% of questions, with only 5.9% and 4.2% solved uniquely by each. Given this high agreement, what evidence shows that the latent states provide reasoning beyond what non-thinking mode already does?
  4. Author report that ThinkBridge solves 74.3% and 93.3% of the questions solved by thinking mode at 0.6B and 4B. What are the corresponding coverage rates for non-thinking mode itself? From Figure F1, non-thinking mode’s pooled accuracy at 4B (about 77.0%) is close to thinking mode’s (about 79.7%), so its coverage may be similarly high. Without this reference, how should the 93.3% figure be interpreted?
Reviewer_4BNrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ThinkBridge, a method designed to reduce the latency associated with explicit Chain-of-Thought generation while preserving the reasoning capabilities of Large Reasoning Models (LRMs). Motivated by the findings that existing fine-tuning-based approaches degrade LRM performance and that the correct predictions from thinking and non-thinking modes are complementary, the authors freeze the backbone model and train only an external reasoner. This reasoner generates compressed latent representations from the hidden states of the input question and is trained using three training objectives. Experiments on five mathematical reasoning benchmarks using Qwen3-0.6B and Qwen3-4B demonstrate that ThinkBridge achieves higher accuracy than existing latent reasoning baselines while maintaining low latency. The authors also show that the method generalizes to multi-turn scenarios without additional training.

Critical strengths

The authors provide a comprehensive analysis of ThinkBridge in both single-turn and multi-turn settings. In particular, they demonstrate its ability to generalize to multi-turn scenarios without additional training.

Critical weaknesses
  1. The experimental results do not sufficiently demonstrate the necessity of using ThinkBridge over the non-thinking or thinking settings. In Tables 1 and 3, ThinkBridge achieves better performance than existing latent reasoning baselines in terms of higher accuracy and lower TTFT. However, it fails to demonstrate a clear advantage over the non-thinking and thinking settings. Compared to the non-thinking setting, ThinkBridge shows only marginal improvements in accuracy or even lower accuracy in some cases, while increasing TTFT. Compared to the thinking setting, ThinkBridge reduces TTFT but also sacrifices accuracy. Therefore, it remains unclear whether ThinkBridge is a preferable choice over the non-thinking and thinking settings when considering both performance and efficiency.
  2. The ablation study results in Table 2 do not provide sufficient evidence that each of the three loss functions makes a meaningful contribution. Although the average results suggest that removing any of the components leads to a performance degradation, the results on individual datasets show that removing certain components can actually improve performance in some cases. These inconsistencies make it difficult to draw a clear conclusion about the effectiveness and robustness of the proposed loss functions.
Questions for authors
  1. Related to Weakness 1. Under what circumstances would ThinkBridge be preferable to the existing non-thinking and thinking modes?
  2. Related to Weakness 2. Could the authors explain why the results in Table 2 are inconsistent across different datasets?
  3. Although the authors describe their approach as latent reasoning, it appears to be similar to existing methods such as prompt optimization or CoT distillation. Could the authors clarify how ThinkBridge fundamentally differs from these approaches?
Reviewer_9r5bOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to train an auxiliary looped model to generate latent reasoning for a frozen model, that substritutes the original reasoning CoT. The resulting ThinkBridge model recovers most of the original base model performance, which was lacking in existing reported latent models.

While the proposed model outperforms other latent reasoning models, it is still behind the thinking version of the base model and matches the non-thinking version but usually at double the cost.

The paper does not compare against existing latent reasoning techniques that keep the backbone frozen, such as softcot and "latent reasoning training".

In terms of efficiency, time to first response token is reported .... but this does not include how long the actual response is, which matters when measuring cost and time incurred.

Critical strengths

The ThinkBridge model recovers most of the original base model performance, which was lacking in existing reported latent models.

Critical weaknesses

While the proposed model outperforms other latent reasoning models, it is still behind the thinking version of the base model and matches the non-thinking version but usually at double the cost.

The paper does not compare against existing latent reasoning techniques that keep the backbone frozen, such as softcot and "latent reasoning training".

In terms of efficiency, time to first response token is reported .... but this does not include how long the actual response is, which matters when measuring cost and time incurred.

Questions for authors

After preliminary training, the model could have been further trained with GRPO rlvr algorithm -- optimizing answer correctness. Was that never tried, or tried but not found to be useful?

Reviewer_NGCmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ThinkBridge, which trains a lightweight external reasoner while keeping the underlying reasoning model frozen. The reasoner maps question hidden states into continuous latent states that condition response generation, bypassing the explicit thinking phase. Training uses the backbone’s successful responses and combines response supervision, on-policy prediction matching against a CoT-conditioned teacher, and a contrastive objective intended to make the latent states question-specific. Experiments on Qwen3-0.6B and Qwen3-4B cover five mathematical benchmarks and three-turn MathChat conversations. ThinkBridge substantially outperforms the evaluated latent reasoning baselines. At 4B, its single-turn accuracy approaches the thinking backbone, although gains over the non-thinking backbone are smaller and do not consistently extend to the other settings.

Critical strengths
  1. The 4B single-turn result is useful: ThinkBridge achieves 84.88% mean accuracy, compared with 85.46% for the thinking backbone, while substantially reducing the time before response generation begins. It does so with additional trainable parameters amounting to approximately 5% of the backbone. This is a practical result for applications where waiting for a long thinking trace is undesirable.

  2. The advantage over the evaluated latent reasoning baselines is large and appears at both model sizes. Appendix D provides an important control: ThinkBridge still outperforms these baselines when trained with dataset-provided supervision. This makes it unlikely that the entire advantage comes from the use of native responses. The paper also reports the original thinking and non-thinking models throughout, making the limits of the improvement visible.

Critical weaknesses
  1. My main concern is that the experiments do not establish the contribution relative to the closest frozen-backbone approach. LRT (Jiang et al., ICLR 2026) already uses an external reasoning network to construct continuous representations for a frozen model. The paper acknowledges this work in Section 5 but does not compare against it. Moreover, ThinkBridge uses filtered native responses, whereas the main trained baselines use dataset-provided targets. Appendix D helps, but it does not answer how a comparable frozen-backbone method, or a strong trained baseline, would perform with the same native supervision. The proposed objectives and compact reasoner may provide a useful improvement, but the current comparisons do not isolate that improvement from the choice of training data and the decision to freeze the backbone.

  2. The evidence that the additional training objectives produce reliable gains is limited. In Table 2, the full objective improves over CE-only by 1.01 percentage points, and removing Match reduces the mean by only 0.36 points. These results use one training seed and are reported only at 0.6B. Relative to the original non-thinking model, the full method is slightly worse in single-turn mean accuracy at 0.6B and improves by 1.09 points at 4B. In the 4B multi-turn setting, accuracy decreases from 75.18% to 74.34%, while mean TTFT increases from 46.8 to 103.8 ms. Thus, the benefit of the added module is not yet consistent across settings. The efficiency evidence also has a narrower scope than the motivation: TTFT ends at the first response token, and the responses in Figures E1 and E2 still contain step-by-step calculations. The paper is explicit about allowing these explanations, but response length and time to the final answer are needed to determine how much computation is actually saved.

I did not find a clear error in the loss formulation. My recommendation is driven by the unresolved contribution and the strength of the empirical evidence. A comparison under matched supervision, together with more reliable evidence for the key ablation gains, could change my assessment.

Questions for authors
  1. What accounts for the improvement over a comparable frozen-backbone method such as LRT when both use the same backbone and native supervision? A controlled comparison on a representative setting would help establish the contribution of the proposed reasoner and training objectives.

  2. Do results from additional training seeds support the full objective’s improvement over CE-only in Table 2? Is the same improvement observed at 4B, where the main single-turn result is strongest?

  3. What are the response lengths, time to the scored final answer, and total generation times for ThinkBridge and the original thinking and non-thinking models? Does the reduction in TTFT also translate into an earlier final answer?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.