acceptodds
Under review as a conference paper at ICLR 2027

CLUE: Contrastive Learning to Unlock Effective Latent Reasoning in LLMs

Abstract

Latent reasoning has emerged as a popular alternative to textual chain-of-thought by carrying intermediate information through continuous representations. As LLMs are pretrained only on text rather than continuous states, post-hoc training is required for them to effectively process latent representations. However, under standard outcome-based reinforcement learning, this inherent text bias can limit the model’s effective use of latent information, as the objective rewards final-answer correctness without explicitly encouraging the model to exploit the information conveyed by the latent representations. We address this challenge by introducing a decoupled architecture paired with a novel training objective. Because latent thoughts represent a fundamentally different knowledge formulation than text, our architecture separates latent generation from textual decoding: a trainable latent generator guides a lightweight, frozen text model, forcing learned adaptation exclusively through the latent channel. To explicitly encourage reliance on these continuous thoughts, we introduce a Contrastive Learning objective to Unlock Effective latent reasoning in LLMs (CLUE). CLUE adds a contrastive loss to the standard GRPO objective, ensuring that correct responses yields a higher log-likelihood under their own positive latent compared to negative latents sampled from other inputs. This provides dense supervision for latent utilization alongside answer correctness, without requiring ground-truth reasoning traces. Trained on OpenThoughts30K and evaluated across diverse mathematical reasoning and scientific question-answering benchmarks, our method improves average accuracy by 5.6 percentage points over direct optimization of the text-based model and outperforms related work by up to 3.5 percentage points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, latent communication, multi-model systems, multi-model collaboration, contrastive learning, counterfactual supervision, latent-utility optimization, reasoning, latent reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_En1VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CLUE, a latent reasoning method. CLUE pairs a trainable latent encoder with a smaller, frozen text model. It adds a contrastive loss to GRPO that favors verified correct trajectories under paired latents over latents from unrelated problems, without requiring ground-truth reasoning traces. Experiments cover four mathematical benchmarks and GPQA, alongside latent-swapping and communication-budget analysis. Results are promising for the 1.7B scale, but mixed at the 4B scale.

Critical strengths
  1. Useful objective with a controlled ablation. Removing the contrastive loss reduces Math500 accuracy from 84.68% to 82.63% and GPQA from 35.04% to 28.91%. This supports a contribution beyond simply adding a larger latent encoder.
  2. Evaluation goes beyond aggregate accuracy. Latent swaps, domain transfers, and reasoning-style probes directly examine whether conditioning affects the receiver. This makes the paper more informative than a benchmark-only comparison.
Critical weaknesses
  1. The latent swap results don't cleanly establish beneficial latent reasoning. Eq. 9 stops gradients through the positive score and explicitly suppresses correct trajectories under mismatched latents. As a result, the paired vs. random gap can just reflect learned sensitivity to mismatches rather than helpful guidance. The Math500 gap is also modest (only 0.60 points).
  2. The advantage doesn't come from fair comparisons. CLUE trains a larger generator, while SoftCoT trains only a small linear projector and HRPO uses LoRA. Direct receiver GRPO also uses a different model capacity and compute budget, and slightly outperforms CLUE on average at 4B scale. Without matched resource comparisons and measured latency / transmitted bytes, the broader superiority and efficiency claims lack empirical backing.
Questions for authors

see weaknesses

2 comments
AuthorsOpenReview ↗

Answer to Reviewer En1V

Thanks for the review and the questions! We address your concerns as the following:

Beneficial Latent Reasoning

The stop-gradient applies only to the paired score within the auxiliary contrastive term. GRPO continues to optimize answer correctness under paired conditioning. Without this detachment, the contrastive objective would update both branches. Our routing deliberately leaves direct paired-response optimization to GRPO while directing the auxiliary gradient through the mismatched score.

Our ablation shows that adding the contrastive objective improves paired task accuracy at fixed architecture and trainable capacity, with gains of up to 6.13 percentage points. These improvements occur under ordinary paired evaluation and cannot be explained solely by depressing accuracy due to sensitivity to mismatches. The observed paired-message advantage is also positive across all datasets evaluated with latent swaps. In contrast the outcome-only baselines show no consistent paired advantage.

To complement these results with a comparison that does not introduce mismatched conditioning. We evaluate a learned, input-independent prefix shared across questions, using the same frozen receiver and latent positions (see Table 2). CLUE exceeds this control by 2.38 and 2.20 percentage points on Math-500 and OpenThoughts30K. This provides additional evidence of task-performance benefits beyond degradation under latent swapping.

Ablation on Problem-Specific Latent Utilization. Accuracy with paired latents and latents generated from randomly selected other evaluation examples. Parentheses report changes relative to paired latents in percentage points.

MethodLatent sourceMath-500OpenThoughts30K
Learned prefixFixed82.3062.40
SoftCoTPaired76.2856.98
Random76.75 (+0.47)57.77 (+0.79)
CLUEPaired84.6864.60
Random84.08 (-0.60)63.40 (-1.20)
CLUE w.o. contrastive lossPaired82.6362.80
Random82.83 (+0.20)63.80 (+1.00)
CLUE w.o. encoder tuningPaired82.1464.29
Random80.03 (-2.11)60.33 (-3.96)

Trainable Parameter Comparison

We added an additional ablation to investigate CLUE under comparable trainable parameters. To this end, we train a CLUE variant following SoftCoTs frozen setup, optimizing only the residual connection and latent input embeddings to the latent encoder (denoted as w.o. encoder tuning in Table 2). Trained on the same data, CLUE w.o. encoder tuning outperforms SoftCoT on Math-500 (82.14 vs. 76.28) and OpenThoughts30K (64.29 vs 56.98). SoftCoT, trained without contrastive supervision, shows no consistent accuracy advantage from using latents generated for the same problem rather than random replacements. Notably, a simple learned prefix also outperforms SoftCoT on both datasets. CLUE retains this advantage across all three datasets even without encoder fine-tuning.

Resource Comparison

Our comparisons match the text-generating model size within each scale. CLUE’s additional, larger encoder constructs the latent prefix in a single forward pass, which the receiver then prefills before generating its answer. Inference time is therefore dominated by autoregressive decoding, rather than by the additional encoder pass. In the profiled CLUE configuration, the encoder takes approximately 0.02 seconds or 1.4% of total inference FLOPs per question, while text generation accounts for nearly all model-call latency.

We additionally report end-to-end latency under native Hugging Face decoding for the 4B encoder and 1.7B text model (Table 5). CLUE achieves comparable measured latency compared to direct GRPO (180.62 s/q vs 178.45). We do not claim a speedup, but interpret more efficient reasoning as similar latency alongside higher accuracy. Frozen-encoder CLUE is the fastest post-trained configuration at 138.11 seconds. The zero-shot comparison shows that GRPO finetuning increases latency across methods.

Estimated training compute is 901.2 PFLOPs for CLUE, compared with 650.7 for direct GRPO and 716.0 without contrastive training. The contrastive objective therefore adds approximately 25.9% training compute relative to the same-architecture ablation. However, contrastive scoring is performed only during training and introduces no corresponding inference-time scoring overhead.

Efficiency comparison for Qwen3-1.7B / Qwen3-4B → Qwen3-1.7B.

MethodTraining compute (est. PFLOPs)Inference latency (s/question)
Qwen3-1.7B-75.38
GRPO650.7178.45
HRPO641.3227.52
SoftCoT686.6148.87
CLUE901.2180.62
CLUE w.o. encoder tuning735.1138.11

Transmission Cost

At comparable task performance in the evaluated configuration, latent communication uses 64 KiB per example versus 8 KiB for text. SoftCoT and CLUE transmit the same latent payload. The communication-budget advantage is therefore fewer sequence positions, not fewer transmitted bytes.

Thanks for the extra results. The small gain from paired latents on Math-500 and the higher training cost still remain a concern. I’m keeping my score at 2 for now.

Reviewer_mWu8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CLUE, a contrastive training objective for latent reasoning with a trainable latent generator and a frozen text decoder. Recognizing that outcome-based reinforcement learning may improve accuracy without encouraging problem-specific latent utilization, CLUE supplements GRPO by suppressing the likelihood of correct responses under latents generated from unrelated inputs. Experiments on mathematical and scientific reasoning benchmarks show improved accuracy over the evaluated latent baselines and increased sensitivity to problem-specific latent information.

Critical strengths
  1. The motivation is intuitive and clear. The paper distinguishes improvements in task accuracy from effective utilization of problem-specific latent information, motivating an explicit training objective beyond final-answer correctness.
Critical weaknesses
  1. The methodological contribution is incremental. The separation of latent generation from textual decoding follows existing frameworks such as SoftCoT and SemCoT, while CLUE primarily adds a contrastive likelihood objective to outcome-based GRPO training. The contribution therefore lies primarily in adapting contrastive training to an established architecture.

  2. The central claim of effective latent reasoning is not sufficiently established. CLUE explicitly reduces the likelihood of correct trajectories under mismatched latents while detaching the positive likelihood, so degradation under latent swapping may partly reflect the incompatibility encouraged during training. In Table 2, paired latents outperform random replacements by only 0.60–1.40 percentage points, with the GPQA difference not statistically significant. These results indicate sensitivity to latent pairing, leaving the contribution of problem-specific information unclear.

Questions for authors
  1. Could the authors provide a sensitivity analysis of the contrastive margin to assess its effect on task accuracy and latent utilization?

  2. Could the authors report end-to-end inference latency and FLOPs under comparable evaluation settings to substantiate the efficiency claims?

2 comments
AuthorsOpenReview ↗

Answer to Reviewer mWu8 (1/2)

Thanks for the review and the questions! We address your concerns as the following:

Main Contribution

We disagree that our contribution is merely a marginal extension of SoftCoT and SemCoT. As acknowledged in the original submission’s Introduction and Architecture subsection, our architectural separation builds on these established frameworks.

Our main empirical finding is that, on Math-500 and OpenThoughts30K, nearly the same accuracy as our outcome-trained latent encoder was attainable without generating a latent message for each question. In our experiments, we find that a GRPO-trained SoftCoT baseline underperforms a learned, input-independent prefix on Math-500 (76.28% vs 82.30%), and OpenThoughts30K (56.98% vs 62.40%). Moreover, replacing its paired latents with unrelated messages produces no consistent accuracy degradation. This shows that generating latents conditioned on the current question does not by itself establish an advantage over generic conditioning. Our experiments expose an alignment problem in latent-reasoning post-training. Final-answer rewards alone do not explicitly require latent messages to provide useful, question-specific guidance.

CLUE addresses these findings through explicit supervision linking correct responses to their paired latent messages. In our experiments, we successfully show that adding CLUEs auxillary objective at fixed architecture and trainable capacity improves paired task accuracy across all reasonnig tasks. We hope that identifying this failure mode will draw greater attention to the alignment challenges in latent reasoning and encourage further work to address them.

Degradation under Latent Swapping and Benefit

Swapping tests whether accuracy changes when the receiver gets a latent from a different question. A drop shows that the choice of latent matters, but it does not tell us how much the paired latent helped or how much the mismatched latent hurt. We have clarified this destinction in our revised manuscript.

The benefit of contrastive training is also visible without swapping. With the architecture and trainable parameters held fixed, adding the contrastive loss improves accuracy under paired conditioning by 2.05 percentage points on Math-500 and 1.80 points on OpenThoughts30K (Table 2). These gains do not depend on a drop in accuracy under swapping.

As an additional alternative to swapping, we also compare CLUE against a learned, input-independent prefix shared across questions. This control represents the limiting case in which a latent encoder becomes completely query-independent and produces the same representations for every question. The prefix outperforms SoftCoT on both mathematical datasets, demonstrating that strong task performance can arise without question-dependent latent messages. CLUE nevertheless exceeds this generic-conditioning control by 2.38 and 2.20 percentage points on Math-500 and OpenThoughts30K, respectively.

Ablation on Problem-Specific Latent Utilization. Accuracy with paired latents and latents generated from randomly selected other evaluation examples. We additionally include an input-independent learned prefix baseline and CLUE ablations without the contrastive loss or encoder tuning. Parentheses report changes relative to paired latents in percentage points. CLUE consistently benefits from paired latents.

MethodLatent sourceMath-500OpenThoughts30K
Learned prefixFixed82.3062.40
SoftCoTPaired76.2856.98
Random76.75 (+0.47)57.77 (+0.79)
CLUEPaired84.6864.60
Random84.08 (-0.60)63.40 (-1.20)
CLUE w.o. contrastive lossPaired82.6362.80
Random82.83 (+0.20)63.80 (+1.00)
CLUE w.o. encoder tuningPaired82.1464.29
Random80.03 (-2.11)60.33 (-3.96)
AuthorsOpenReview ↗

Answer to Reviewer mWu8 (2/2)

End to End Inference Latency

We report measured end-to-end inference latency and estimated training FLOPs in Table 5. Latency includes latent construction and answer generation, averaged across the four mathematical benchmarks under the common inference protocol described in the manuscript.

For the Qwen3-1.7B text model, full CLUE achieves higher task accuracy than direct GRPO with similar measured latency. The contrastive objective increases estimated training compute by 25.9% relative to CLUE without the contrastive loss but adds no inference-time scoring overhead. CLUE w.o. encoder tuning has the lowest measured latency among the tested methods. CLUE’s Qwen3-4B latent encoder and projection together require about 0.99 TFLOPs per question, or 1.4% of total inference FLOPs. The remaining compute is spent on text generation. These results support an accuracy–cost trade-off rather than a uniform token efficiency advantage.

Efficiency comparison for Qwen3-1.7B / Qwen3-4B → Qwen3-1.7B. Inference latency is averaged across the evaluated mathematical reasoning benchmarks using native HuggingFace decoding.

MethodTraining compute (est. PFLOPs)Inference latency (s/question)
Qwen3-1.7B-75.38
GRPO650.7178.45
HRPO641.3227.52
SoftCoT686.6148.87
CLUE901.2180.62
CLUE w.o. encoder tuning735.1138.11

Sensitivity Analysis of Contrastive Margin

We add an additional ablation varying the weight of the contrastive margin in {0.0005,0.002,0.008}. Each configuration uses the same pretrained Qwen3-4B/Qwen3-1.7B pairing, using 16 latent positions and shared training configuration. We evaluate Avg@8 accuracy on Math-500, OpenThoughts30K, and GPQA to assess its influence on final task accuracy. To assess latent utilization under the different configurations, we additionally report accuracy using latents generated from randomly selected other evaluation examples, similar to Table 2.

Resuls are reported in Figure 4 in the appendix. We find that 0.00050.0005 achieves the highest paired accuracy across all three datasets. Increasing the weight enlarges the paired-control gap but reduces task accuracy. This suggests that stronger contrastive weighting can increase dependance on latent conditioning at the expense of task performance and that a larger control gap does not necessarily indicate more effective latent reasoning.

Reviewer_KnqyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a contrastive loss to directly reward latent states for a query that are far from the latent states generated for other queries. In particular, it proposes a variation of softcot that uses a larger model to produce CoT for a smaller model, with two variations, one, is that they train the parameters of the larger model for latent CoT generation and two, they add a contrastive loss to make sure that different querys endup produceing different latent representations.

Critical strengths
  • The work points to an important issue when generating latent CoT i.e. lack of direct supervision for intermediate latent representations.
  • The results show that using contrastive loss actually improves results over its grpo only ablation and also makes the latent states associate better with their queries. (table 2)
Critical weaknesses
  • The paper shows limited gains compared to SoftCoT when that method is not even tuning the parameters of the latent generator network. Using both models in their original frozen state is a positive component of SoftCoT architecture making it more generalizable. If CLUE instead trained the whole latent generator model, then the approaches are no longer comparable in terms of trained params. And despite that advantage in params, CLUE still does not always outperform.
Questions for authors
  • it is unclear to me why the contrastive objective gradients only flow to negative examples. It seems more intuitive that they only flow to positive ones so that they can be far from other representation in the direction of correct trajectory ... however the trajectory specific nature of this loss can overfit to one generated trajectory ... which might be less desirable than the already applied GRPO loss. Still it will be interesting to see if "no stop gradient" and "stop gradient for only negative" variations were tested too.
1 comment
AuthorsOpenReview ↗

Answer to Reviewer Knqy

Thanks for the review and the questions! We address your concerns as the following:

Training Budget and Generalization

We agree that our original comparison with SoftCoT changes both the training objective and which parameters are trained. The performance difference thus cannot be attributed to the contrastive loss alone. To this end, we added a CLUE variant following SoftCoTs frozen setup, training only the residual connection and latent input embeddings to the latent encoder (denoted as w.o. encoder tuning in Table 2). Trained on the same data, CLUE w.o. encoder tuning outperforms SoftCoT on Math-500 (82.14 vs. 76.28) and OpenThoughts30K (64.29 vs 56.98). Notably, a simple learned prefix also outperforms SoftCoT on both datasets (Table 2). This prefix uses the same learned vectors for every question and requires no latent encoder.

Ablation on Problem-Specific Latent Utilization. Accuracy with paired latents and latents generated from randomly selected other evaluation examples. We additionally include an input-independent learned prefix baseline and CLUE without encoder tuning. Parentheses report changes relative to paired latents in percentage points. CLUE consistently benefits from paired latents.

MethodLatent sourceMath-500OpenThoughts30K
Learned prefixFixed82.3062.40
SoftCoTPaired76.2856.98
Random76.75 (+0.47)57.77 (+0.79)
CLUEPaired84.6864.60
Random84.08 (-0.60)63.40 (-1.20)
CLUE w.o. encoder tuningPaired82.1464.29
Random80.03 (-2.11)60.33 (-3.96)

Why Detach the Positive Branch?

GRPO trains the model to produce correct solutions under paired latents, already encouraging higher likelihood for successful trajectories. The contrastive loss complements this by reducing the likelihood of the same correct trajectory under latents from a different question. We therefore detach the paired score only within the contrastive term, leaving direct optimization of paired responses to GRPO.

We add an additional ablation comparing the stop gradient position in the contrastive loss. CLUE clearly achieves the best or joint-best accuracy on all four mathematical benchmarks, e.g., 89.10 vs. 88.37 (no sg.) and 87.94 (sg. on neg.).

Effect of stop-gradient placement in the contrastive loss. We report Avg@8 accuracy (%) for the same-architecture GRPO-only baseline and three gradient-routing variants.

VariantGSM8KMath-500AIME24AIME25
w.o. contrastive loss88.0382.6334.5824.17
No stop-gradient88.3783.1832.9225.42
Stop-gradient on negatives87.9482.7331.6722.50
Stop-gradient on positives (CLUE)89.1084.6834.5827.08

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response - thanks to all reviewers for their valuable feedback

We’d like to thank all reviewers again for their valuable feedback. In the following, we will use R1-R3 to refer to the reviewers in the order of their reviews.

We are pleased that R1 highlighted the lack of direct supervision for intermediate latent representations as an important problem. R2 found our motivation “intuitive and clear” and valued the distinction between answering a question correctly and using latent information specific to that question. R1 and R3 both highlighted the controlled ablation showing that contrastive training improves accuracy over GRPO alone. R1 also noted a stronger association between latent messages and their questions. Finally, R3 appreciated that our latent swaps, domain transfers, and reasoning-style probes examine how latent conditioning affects the text model beyond what aggregate accuracy can show.

We summarize the main revisions and additional results below.

  1. Question-specific conditioning (R2, R3). As an alternative to latent swapping, we added a learned prefix shared across all questions. On math reasoning benchmarks, this simple prefix outperforms SoftCoT, whereas CLUE improves upon both.
  2. Stop-gradient placement (R1). We clarified why CLUE detaches the positive score in the contrastive loss and provide additional experiments identifying this choice as best or tied for best across stop-gradient positions.
  3. Capacity and resource costs (R1, R2, R3). We added a CLUE ablation with both backbones frozen. This variant outperforms SoftCoT on mathematical reasoning benchmarks. In addition, we report measured end-to-end inference latency, estimated training compute, and bytes transmitted per example.
  4. CLUE’s contribution (R2). We revised the Abstract and Introduction to clarify our contribution, that is identifying a limitation of final-answer supervision and introducing an objective that addresses it.
  5. Contrastive loss sensitivity (R2). We provide additional results for an analysis on the margins hyper-parameter.

We thank the reviewers again for helping sharpen the paper’s central distinction. Our controlled comparisons show that latent reasoning models can achieve strong task accuracy even under generic conditioning. We identify a limitation of final-answer supervision and show how contrastive training can help address it. We hope these findings will spark further work on learning latent messages that provide useful, question-specific guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.