acceptodds
Under review as a conference paper at ICLR 2027

What Matters for Latent Reasoning with Flow Matching

Abstract

Latent reasoning lets a large language model (LLM) think in a continuous space and verbalize only the answer. We argue that an effective latent thought must meet five requirements: it should be useful, helping produce the correct answer rather than merely changing it, diverse, so that resampling yields different reasoning trajectories, explainable, so that a decoded chain of thought (CoT) reflects reasoning the answer actually follows, refinable with more inference compute, and efficient, costing less than an explicit CoT at comparable accuracy. Current methods rarely meet these requirements: they learn shortcuts from the question, distill the explicit CoT into their weights, or imitate it one token at a time. We focus on flow matching in a learned latent space, the family we argue is best placed to meet them, and identify the training choices that make it work. The result is Flow-based Latent Reasoning (FLaRe), a simple recipe covering what the latent space encodes and how to shape it, where to train the flow, how to read out the answer, and a final stage of training on the model's own verified thoughts. A probe for each requirement shows that FLaRe improves on prior latent methods in all five. It also compares favorably with them on arithmetic benchmarks, while reaching 97% of the accuracy of explicit CoT at a quarter of its latency.

Primary area:
generative models
Keywords:
latent reasoning, continuous chain of thought, flow matching, latent diffusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6j5kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FLaRe, a latent reasoning approach based on flow matching. A VAE encodes symbolic chains of thought into compact latent codes, while a shared model learns to generate these codes from a question and use them to predict the final answer. The model is trained to read its own imperfectly generated codes and is further optimized through verified self-training. The authors evaluate the resulting latent thoughts in terms of usefulness, diversity, explainability, refinability, and efficiency.

Critical strengths
  1. The paper presents a coherent end-to-end recipe for flow-based latent reasoning and evaluates latent thoughts along five dimensions beyond answer accuracy.
  2. The experimental study is extensive. The authors compare FLaRe with several latent reasoning baselines and report the five proposed diagnostic measures. They also ablate code capacity, VAE corruption, symbolic and dual-route targets, flow time sampling, and the inputs used to train answer generation. These results help identify which choices affect the reported accuracy.
Critical weaknesses
  1. The paper does not clearly distinguish FLaRe from LaDiR, which already uses a VAE, flow matching, a shared model for latent generation and answer prediction, and rollout-based training. FLaRe's main architectural difference is to generate a single code for the entire CoT rather than a sequence of latent blocks. The comparison in Table 8 changes several representation and training choices together, so it cannot establish how much of the gain comes from this difference. This makes the core novelty and its contribution to performance difficult to assess.
  2. The link between the five proposed criteria and the training recipe is not established clearly. The ablations in Section 3 mainly report direct and decoded answer accuracy; they do not show whether each training choice improves the property it is intended to support. Thus, the evidence for systematically addressing all five criteria remains incomplete.
Questions for authors
  1. Why generate an entire reasoning chain in one latent block rather than generate blocks sequentially as in LaDiR? Sequential blocks can naturally accommodate longer and variable-length reasoning, whereas a single code must hold the full chain. Appendix A.4 considers variable-width codes on GSM8K, whose symbolic CoTs are short and already fit in eight slots. Is there evidence that the accuracy and efficiency advantages persist on substantially longer reasoning traces?
  2. What motivates the selection of these five properties as requirements for effective latent reasoning, rather than as task-dep
5 comments
AuthorsOpenReview ↗

Response (part 1)

We thank the reviewer for their detailed review and their suggestions. The revised paper positions FLaRe against LaDiR, shows that our recipe wins on LaDiR's blocks as well as on our one code, scopes the claims to GSM8K-length reasoning while showing how the recipe extends to longer traces, and links each recipe choice to the five requirements with probes.

Below, we answer each weakness (W) and question (Q) in turn, together when they overlap. Two concerns shared with other reviewers are answered once in the general response, whether the gains come from the recipe or from the data (G1) and how FLaRe differs from LaDiR (G2), and we refer to it for W1 and Q1.

AuthorsOpenReview ↗

Response to W1 and Q1 (part 2)

W1. Distinction from LaDiR. We thank the reviewer for this point, and the revision now positions FLaRe clearly against LaDiR. Please refer to the general response (G2) for the shared foundations, the measured choices, and both recipes trained on both layouts.


Q1. Why one code rather than sequential blocks? We thank the reviewer for this question. On GSM8K, one code is both more accurate and faster than a sequence of blocks, and it keeps the study focused on how to train the flow:

  • More accurate. With the same recipe, one code beats LaDiR's blocks by 16.0 points with our recipe (56.6 against 40.6) and by 19.8 with LaDiR's (41.3 against 21.5, please refer to G2). Blocks add errors that carry over from one block to the next, and a decision of when to stop. Against LaDiR's full method, the lead of one code holds at every number of reasoning steps, and is largest in relative terms on the longest chains (table below).
  • Faster. One code is denoised once for the whole thought, so its sequential passes are set by the number of flow steps. Blocks need a full denoising per block, so their passes grow with the length of the reasoning.
  • Focused. With one code, every ablation measures the recipe alone, without the extra choices that blocks bring (block size, when to stop, how each block conditions on the earlier ones), which could confound the takeaways on how to train the flow.
Reasoning steps123456+
Questions3835639429214891
LaDiR, 1B34.264.343.131.220.92.2
FLaRe, 1B60.578.168.049.342.615.4

We also note that none of the recipe's choices assumes a fixed width, and Appendix A.4 trains a variable-width code with almost no change, though with no gain on GSM8K since eight slots already give almost 100% exact reconstruction. We discuss much longer traces below.

Q1. Longer reasoning traces. GSM8K chains are short, so we cannot yet show directly that the advantages persist on much longer traces, and the revision scopes its empirical claims to GSM8K-length reasoning. Three results nevertheless point that way:

  • Accuracy within GSM8K. On the longest chains the code still reconstructs almost every CoT exactly (93.8% above 40 tokens, table below), so accuracy falls because the questions get harder, not because the code runs out of room. FLaRe's lead over LaDiR is also largest on the longest chains (table above).
  • Accuracy beyond GSM8K. A much longer trace would be split into a sequence of codes, generated one after the other by the same flow, as in LaDiR and in latent and block diffusion models of long text [1, 2], rather than held in one wider code, since more slots lower accuracy even on GSM8K (Tab. 1). Our recipe already transfers to this layout without change, and lifts LaDiR's blocks by 19.1 points (G2). None of its choices refers to the length of the trace.
  • Efficiency. FLaRe's speed advantage depends on how many tokens each code replaces, not on the length of the trace. Within GSM8K, it spends the same passes on every question, while explicit CoT spends one per token. On longer traces, the advantage persists as long as each segment holds more tokens than the flow takes steps, as on GSM8K, where one or two steps already make FLaRe about 5× faster at 1B.
Gold CoT tokens≤1516–2021–2526–3031–4041+
Exact reconstruction100.0100.0100.099.198.693.8
Direct accuracy80.071.871.353.243.320.1
  • [1] Barrault et al. Large concept models: Language modeling in a sentence representation space. 2024.
  • [2] Arriola et al. Block diffusion: Interpolating between autoregressive and diffusion language models. 2025.
AuthorsOpenReview ↗

Response to W2 and Q2 (part 3)

W2 and Q2. The five requirements and the recipe. We thank the reviewer for these questions. The five requirements are not tied to FLaRe: they follow from what any latent method changes with respect to explicit CoT, so they apply to the whole family of latent reasoning methods, ours included. The revision derives them first, then links them to the recipe with probes.

Why these five (Q2). Explicit and latent reasoning both draw a thought r∼pθ(r∣q)r\sim p_\theta(r\mid q) and answer from it, a∼pθ(a∣q,r)a\sim p_\theta(a\mid q,r). They differ only in what rr is: written tokens, or a continuous vector that is never verbalized. The requirements are what this change must gain and what it must not lose: it must gain efficiency, the reason to leave the vocabulary, and keep the four properties explicit CoT has by construction:

RequirementIn explicit CoTFor a latent thought
Efficientone pass per tokenit costs less at a similar accuracy
Usefulthe answer reads the written stepsthe answer depends on it
Diverserr is sampledit remains a distribution
Refinablea longer chain spends more computemore compute improves it
Explainablethe steps are in languageit decodes to the CoT the answer follows

Useful is the precondition, since a thought the answer ignores brings none of the others. Since the five follow from the definition of latent reasoning and not from a task, they do not depend on the task, only the probes that measure them are specific to arithmetic.

From the requirements to the recipe (W2).

  • Why flow matching. Prior latent methods gain efficiency but lose much of the rest (Sec. 4): their thoughts can be removed or swapped, each question gets one thought, the budget is fixed at training, and nothing reads the thought back. Flow matching in a VAE latent space gives four requirements by design: noise draws give diversity, denoising steps refinement, the decoder explanations, and a small code refined in parallel efficiency. What it does not give is a useful and accurate thought, and this is what our recipe provides (Sec. 3).
  • Requirements, not training objectives. We do not optimize them directly, and since their weight depends on the application, we report them separately rather than as one score.
  • Each recipe choice, probed. As the reviewer asks, we now run all five probes on the main ablation arms of Sec. 3, not only accuracy:
1B armUsefulDiverseExplainableRefinableEfficient
Final stage 139.7+16.646.0+0.4 / +18.282%
Final stage 238.3+14.948.7+2.0 / +23.585%
Exact-code answer pass32.0+11.945.0−0.5 / +20.269%
Uniform timeᵃ22.6+24.039.7−1.2 / +2.047%
Single CoT per question (D0)36.5+11.843.7−0.6 / +17.979%
Single reconstruction route31.8+12.542.9−1.9 / +9.575%

Compared with final stage 1, removing any one of these choices makes the thought less useful, less explainable and less efficient. The two exceptions are on the decoded reading: uniform time samples more widely, from a much lower greedy accuracy (48.8), and the exact-code arm gains slightly more from steps.

Reviewer_nJjDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a flow-matching based method for latent reasoning with LLMs, where a pretrained LLM is further trained to execute the reasoning trace via an iterative refinement of latent code that is conditioned on the query, followed by auto-regressive answer generation. The paper provides a detailed study on the important components in training and designing such models which motivate the final design, followed by a comparison to standard baselines on math benchmarks and additional analysis on important aspects of latent reasoning.

Critical strengths
  1. The work provides a highly detailed study on the proposed design, demonstrating the benefits of each stage in a rigorous fashion.
  2. The use of flow matching for latent reasoning is novel and non trivial, providing a method for latent reasoning with adaptive compute, which is typically lacking and is better aligned with text-based CoT.
Critical weaknesses
  1. The design space contains several components that are not unique to FLaRe, while the comparison between direct and decoded answer is fair, it is unclear whether the considered baselines follow an equivalent procedure. More specifically, further augmenting the dataset, as in section 3.2 Table 5, or training on model generated rollouts, as in stage 2, will likely lead to gains in most methods, especially the CoT model. While these lead to different compute requirements, it does not seem like training compute is controlled in the experiments.
  2. As the main contribution of the paper in my opinion is the detailed study of the design, a comparison on whether the gains from the different steps are compounding or redundant is missing. E.g. does stage 2 make the diversification of the data redundant or more important?
Questions for authors
  1. Can you provide an analysis comparing the effect of data diversification and training on rollouts for some of the baselines. Alternatively, since this might be expensive, can you provide a comparison in a FLOP-matched setting between FLaRe and other methods?
  2. Is it possible to identify which of the design choices are important in context of the full method?
3 comments
AuthorsOpenReview ↗

Response (part 1)

We thank the reviewer for their detailed review and their suggestions. As asked, the revised paper retrains the baselines with our data and rollouts rather than matching floating-point operations (FLOPs), discusses training compute, tests whether Diverse CoT and stage 2 compound, and makes the presentation easier to follow.

Below, we answer each weakness (W) and question (Q) in turn, together when they overlap, and end with the presentation. Two concerns shared with other reviewers are answered once in the general response, whether the gains come from the recipe or from the data (G1) and how FLaRe differs from LaDiR (G2), and we refer to it for W1 and Q1.

AuthorsOpenReview ↗

Response to Ws & Qs (part 2)

W1 and Q1. Data and rollouts for the baselines, training compute. We thank the reviewer for this suggestion. We ran the full analysis rather than the FLOP-matched alternative: every baseline is retrained with our Diverse CoT and with a stage 2 on rollouts, at three scales (details in G1). As the reviewer expected, explicit chain of thought (CoT) gains from both (+2.5 to +6.0 points from D0 to S2). The latent baselines gain far less (at most +4.3 points), while FLaRe gains 6 to 8 points and leads every latent method on D1 and S2. As for the training compute:

  • FLaRe's training is indeed heavier. It trains a variational autoencoder (VAE) and then a flow in two stages, while each baseline fine-tunes its backbone once.
  • A FLOP match on fine-tuning alone would favor the baselines. The latent baselines keep the autoregressive interface of their backbone, so their low fine-tuning cost rests on the pretraining and instruction tuning that already taught the backbone to generate and to reason (G1). Our flow cannot reuse that interface and must be trained from scratch to turn noise into codes of a new latent space, so its training pays for a generator that the baselines received from pretraining. A fair compute comparison would therefore have to count the pretraining and instruction tuning behind each baseline as well, and against that budget the extra cost of the VAE and the flow is small.
  • The training cost is paid once, while the savings at inference, fewer sequential passes per question, apply to every question.

W2 and Q2. Do the gains compound? We thank the reviewer for this question. The gains compound: removing any single choice from the full method lowers accuracy, so no choice is made redundant by the others. We show this in two ways, and explain why below:

  • Removing one choice at a time (Q2). From the final stage 1 we remove one choice and keep all the others. Two-step direct accuracy falls from 82% of explicit CoT's (S2) to 79% without Diverse CoT (D0), 75% with a single reconstruction route, and 69% with an answer pass trained on exact codes only. The paper's ablations show the same within each part: dropping any one VAE corruption lowers decoded accuracy (Tab. 2), and in stage 2 the gain comes from the rollout answer loss and verification (Tab. 6).
  • Diverse CoT and stage 2. Stage 2 makes Diverse CoT more important, not redundant: stage 2 adds 3.4 points on top of Diverse CoT, against 2.5 without it (table below, 1B, D0: GSM8K-Aug, D1: plus Diverse CoT). Stage 2 trains on the correct thoughts the model samples, so it benefits from varied samples, and Diverse CoT, with several CoTs per question, teaches the model to produce them. For explicit CoT the two overlap instead: rejection-sampling fine-tuning (RFT [1]) adds 1.3 points after Diverse CoT, against 2.7 from D0.
  • Why the gains do not overlap. Each choice fixes a different failure. The VAE choices decide what the flow must generate, and the flow choices decide how it is generated and read. Sec. 3 is sequential, so each choice is measured on top of all earlier ones, and decoded accuracy keeps rising along the way (from 53.2 to 62.6).
1BD0D0 + RFTD1D1 + stage 2 / RFT
Explicit CoT61.664.366.367.6
FLaRe (ours)52.254.756.660.0
  • [1] Yuan et al. Scaling relationship on learning mathematical reasoning with large language models. 2023.

Presentation. We revised the paper to make it easier to follow: it now opens the ablations with a summary of the final recipe, states the setting and reading behind every result, compares all methods in one place under matched data, and orders the appendix to follow the main text. We would be grateful for any other passage the reviewer found hard to follow.

I thank the authors for their thoughtful response which address most of my concerns, I am increasing my score.

Reviewer_QBHnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents FLaRe, a latent reasoning framework that uses flow matching to transform Gaussian noise into a continuous reasoning representation used to generate the answer with the same language model. Its novelty lies primarily in a systematically evaluated training recipe rather than the general use of latent diffusion for reasoning, which builds on prior work such as LaDiR. The recipe combines symbolic CoT compression with dual reconstruction objectives, latent-space corruption, noise-focused flow training, answer prediction from imperfect latents, and end-to-end training through the full rollout. The paper also proposes five requirements for latent reasoning and corresponding evaluation probes. The evaluation compares FLaRe with existing methods across three backbone sizes on multiple arithmetic benchmarks: GSM8K, GSM8K-Hard, SVAMP and MultiArith. Extensive ablations examine which training choices contribute to performance. The paper distinguishes direct answer generation from evaluation through a separate CoT decoder. In the reported latency comparison, a two-step direct-inference configuration achieves 57.4% accuracy versus 59.3% for explicit CoT, while reducing latency from 267 ms to 68 ms.

Critical strengths

The extensive ablations provide actionable insights into latent reasoning. The study shows that high VAE reconstruction accuracy does not guarantee effective downstream reasoning and that training the answer reader on generated, imperfect latents substantially improves direct accuracy. These findings make the paper informative beyond the particular FLaRe implementation and constitute its strongest scientific contribution.

Critical weaknesses
  1. The practical benefits of diversity and refinement are less established for direct inference than for decoded CoTs. For stage-2 direct answers, sampling improves oracle coverage to 66.2% pass@16, but majority voting achieves 58.2%, compared with 59.1% for a single fixed sample (Table 16). Increasing denoising from one to twenty steps improves direct accuracy from 56.7% to 59.1%, while latency increases from 47 ms to 461 ms (Table 19). These results demonstrate additional coverage and some refinement, but the stronger benefits rely on a separate autoregressive decoder. The paper should therefore distinguish improvements in latent decodability from practical accuracy gains in the efficient direct-answer system.
  2. The comparisons leave the source and size of the proposed method’s advantage partly unresolved. Most baselines do not receive FLaRe’s additional synthetic traces and stage-2 questions, so the results do not fully separate improvements from the training recipe and additional supervision. The claim “97% of CoT accuracy at a quarter of its latency” is valid for the measured CoT checkpoint, which scores 59.3%. However, the authors’ own explicit-CoT baseline with the same Llama-3.2-1B-Instruct backbone scores 62.5% in Table 8. Two-step FLaRe achieves 57.4%, or 91.8% of that baseline’s accuracy, but its latency is not reported. Comparisons with matched supervision and timing of the authors’ higher-scoring CoT baseline would make the claimed advantage more convincing.
Questions for authors
  1. All evaluated backbones are 0.5B–3B models from the Llama-3.2 and Qwen2.5 families. What motivates this choice, and do you have results on newer model families, larger models, or models specifically trained for reasoning? Such evidence would help establish whether the recipe’s benefits and its accuracy/latency advantage over explicit CoT extend beyond the evaluated backbones.
  2. The paper promises to release code, data, and checkpoints, but I could not find any supplementary materials. Access to the code and configurations would help assess reproducibility.
  3. Appendix E.5 states that each baseline uses its reference inference code. Which inference frameworks and optimizations were enabled for FLaRe and the baselines? Fast inference frameworks like vLLM or SGLang can drastically change latency and relative speedups, so were comparable inference optimizations used across methods? Could you also report throughput and latency at larger batch sizes, since single-question latency alone does not establish efficiency under batched inference?
  4. As far as I understand, all reported benchmarks involve math problems. Do you have evidence that the main findings extend to other reasoning tasks, such as logical deduction or planning, or should the conclusions be understood as specific to arithmetic reasoning?
  5. Could you address the concerns raised under Critical Weaknesses, particularly the distinction between direct and decoded results, differences in training supervision across baselines, and the choice of explicit-CoT checkpoint for the latency comparison?
7 comments
AuthorsOpenReview ↗

Response (part 1)

We thank the reviewer for their detailed review and their suggestions. The revised paper now reports the direct and decoded readings separately, fixes the inconsistent numbers, compares with the explicit CoT trained on the same data, times every method at several batch sizes, and adds a 7B model as well as logic and planning tasks.

Below, we answer each weakness (W) and question (Q) in turn, in the order of the review. Two concerns shared with other reviewers are answered once in the general response, whether the gains come from the recipe or from the data (G1) and how FLaRe differs from LaDiR (G2), and we refer to it for W2.

AuthorsOpenReview ↗

Responses to W1 and W2 (part 2)

W1. Diversity and refinement for the direct answer. We agree that the two readings must be kept apart and we do so in the revision. The table reads the same 16 samples of our final 1B model in both ways:

Direct answerDecoded thought
Stage 2, 1B, 16 samplesOne sampleVotePass@16One sampleVotePass@16
Accuracy57.658.866.962.265.478.0

One sample: accuracy of a single sample averaged over k=16.

On the direct answer, a vote adds 1.2 points over one sample, and on the decoded thought 3.2, reaching 65.4. As for the raised points:

  • Sampling gives real coverage. Pass@16, which is the share of questions solved by any of 16 samples, reaches 66.9 against 57.6 for one sample (table above), and this gain is close to that of explicit chain of thought (CoT) and exceeds every latent baseline.

  • Voting is not the right measure of diversity. A vote only helps when the correct answer is already the most common one among the samples. A diverse model spreads its samples over several answers, so even when one of them is correct, it is often not the most common. This is especially true on GSM8K, where the answer is a single number and wrong samples often agree on the same value. Voting therefore gains little for every method, explicit CoT included (+0.9 with 32 votes), and the plain vote stays close to a single sample (table above). What diversity provides is coverage, the chance that at least one sample is correct, which is what pass@16 measures.

  • Diversity turns into accuracy through training. Because the samples differ, one of them is often correct even when the first one is wrong. Our stage 2 uses exactly this: it samples several thoughts for new questions, keeps the correct ones and trains on them. These extra correct samples double the gain of stage 2 (+3.8 against +1.9 without them), which validates that FLaRe does indeed produce diverse thoughts that are also useful.

  • The direct answer needs little refinement, by design, for efficiency. We want a fast answer, so we train the answer pass to read rough, one-step thoughts, much like step distillation for diffusion models [1], so it is near its best after one step: 57.0 at 36 ms, against 59.9 at 183 ms with 20 steps and a key-value (KV) cache. GSM8K's short answers also limit refinement, since the answer pass only has to produce a number. The practical gain from more steps is therefore modest on this reading, and one or two steps are enough.

  • The decoded gains are practical too. Diversity and refinement are properties of the thought, and the decoded reading measures them directly: accuracy rises from 36.5 to 63.1 with steps, and pass@16 adds 14.9 points. While it requires running the decoder, this is still useful whenever the full thought is needed rather than a short number as when inspecting or verifying the reasoning, and it could extend to longer outputs such as code. This also highlights the flexibility of FLaRe: one model gives either a fast direct answer or a full thought that improves with sampling and steps, a choice that carries over to any task where the reasoning itself matters.

  • [1] Salimans and Ho. Progressive distillation for fast sampling of diffusion models. 2022.


W2. Matched supervision. We thank the reviewer for raising this. Please refer to the general response (G1), where we retrain every baseline on our data and rollouts under one protocol.

W2. The CoT checkpoint for the latency claim. We thank the reviewer for pointing this out, and agree that our efficiency claim should be measured against our own, stronger explicit CoT. The 97% figure compared FLaRe with the released checkpoint of Dilgren et al. [1], which we had used for the analysis. The revision replaces it with uniform numbers: one FLaRe model (our final stage 2), explicit CoT from our own runs, the same evaluation for all, and absolute latencies next to every ratio, all on one H100 with a KV cache for every method (at batch 1, FLaRe takes 36 ms at 1 step and 69 ms at 5 steps, against 185 ms for explicit CoT). On this basis, FLaRe keeps 84% (1 step) to 87% (5 steps) of the accuracy of explicit CoT trained on the same data, at 5× to 2.7× its speed in the same stack (93% to 96% against explicit CoT trained on GSM8K-Aug only).

AuthorsOpenReview ↗

Reponses to Q1 and Q2 (part 3)

Q1. Backbones. We thank the reviewer for this question. We mainly focused on these backbones because prior latent methods report results on them (CODI at 1B, KaVa at all three), which allows a direct comparison with published numbers, and because their size keeps full ablations affordable. We address the other points below, namely larger models, newer families and reasoning models, and whether the recipe and its advantage over explicit CoT extend:

  • Larger models. We add Qwen2.5-7B-Instruct on GSM8K, with the data versions of G1. As at 1B and 3B, FLaRe leads every latent method on D1 and S2 (61.6 and 64.7, against 42.2 and 61.6 for CODI) and only lags CODI on D0 (59.4 against 59.9).
  • Newer families and reasoning models. We thank the reviewer for the suggestion. They matter less for our study, for two reasons. First, our contribution is a recipe for training the flow, and the flow learns its latent space from scratch instead of reusing the backbone's reasoning (G1), so the findings should depend little on the backbone, as our two families and four scales show. Second, reasoning-tuned models write long native traces, and compressing them into latent thoughts is the question of longer traces, separate from how to train the flow. We plan both as future extensions.
  • Do the recipe's benefits extend? The recipe behaves consistently over two families and four scales, and stage 2 helps at every scale (+2.4, +3.4, +2.2 and +3.1 points from 0.5B to 7B).
  • Does the advantage over explicit CoT extend? In accuracy, explicit CoT remains the upper bound at every scale (G1) leveraging the same natural language form of reasoning the backbone saw at scale during pretraining and instruction tuning. FLaRe's latent space on the other hand introduce a new interface to the pretrained LLM and it only learned from our fine-tuning data. FLaRe and other latent methods therefore aim at other axes that leverages the continuous space rather than in discrete tokens. These advantages come from the design rather than the backbone, and we expect them to carry over to other models:
    • Speed. The number of sequential passes is set by the number of flow steps on any backbone, while for explicit CoT it grows with the length of the reasoning. For 1B backbone: in the same PyTorch stack, FLaRe is about 5× faster than explicit CoT at one or two steps.
    • Diversity. Each sample draws a whole new thought from noise, so 16 samples solve 66.9% of questions against 57.6% for one sample (W1), a gain close to explicit CoT's and above every latent baseline.
    • Flexibility. A single model gives either a fast direct answer or a decoded thought, and its accuracy grows with more samples and steps, so test-time compute can be traded for accuracy. Its diverse samples also make it ready for more elaborate post-training, such as reinforcement learning.

Q2. Code. We apologize for the missing material. Our institution requires an internal approval before release, which could not complete before the deadline. We will release the code, data, probes and checkpoints once it does. Meanwhile, Tab. 9, Appendices A.2 to A.3 and Algorithms 1 to 3 give every hyperparameter and step, and we are happy to answer any implementation question.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response (part 1)

We thank the reviewers for their careful reading and for their helpful and thoughtful reviews. They found: the ablations extensive and actionable (QBHn), the study rigorous and flow matching for latent reasoning novel and non-trivial (nJjD), and the end-to-end recipe coherent and informative (6j5k). Following their comments, we have revised the paper throughout, with new experiments and clearer explanations. The main changes are:

  • Matched data: all methods retrained on the same data with a clearer explanation of why FLaRe needs Diverse CoT and stage 2.
  • New experiments: logic and planning tasks, and more probes.
  • Latency: every method timed in the same setup, at batch sizes 1 to 32.
  • Two readings: direct and decoded accuracy reported separately.
  • Relation to LaDiR: what FLaRe shares with it and what it adds, in detail in the related work and appendix.
  • The five requirements: where they come from and why we chose them, explained directly.
  • Claims & Numbers: restated to match the new results.
  • Clarity: the final recipe and the setup of each table made easier to find, and unclear passages rewritten.

Below, we first answer the two concerns shared by several reviewers: whether the gains come from the recipe or from the extra data (G1), and how FLaRe differs from LaDiR (G2). We then answer each reviewer separately.

AuthorsOpenReview ↗

General Response 1 - G1: results on matched data (part 2)

To separate the recipe from the data, we retrained every baseline with its public code on our data, with one training and evaluation protocol shared by all methods over three data versions. With matched data, FLaRe's final model remains the most accurate latent method at every scale (table below). The three versions are:

  • D0: the original GSM8K-Aug (385K) baseline data.
  • D1: D0 plus our 216K Diverse CoT rows with several chains of thought (CoTs) per question.
  • S2: a second stage on top of D1 using our 100K stage 2 questions and keeping only traces that reach the correct answer. For a fair comparaison, we train each method with the closest match to our stage 2:
    • Explicit CoT samples its own CoTs and trains on the correct ones (rejection-sampling fine-tuning or RFT [3]). For No-CoT, we train it on full 100K questions and their ground truth answers.
    • Coconut, iCoT and CODI produce one fixed thought per question, so there is nothing to sample. They train on the correct CoTs obtained using explicit CoT.
    • LaDiR runs its own stage 2.
    • FLaRe uses our proposed stage 2, self-training on its own verified rollouts: it samples latent thoughts from noise and trains on those that reach the correct answer.
Qwen2.5-0.5BLlama-3.2-1BLlama-3.2-3B
MethodD0D1S2D0D1S2D0D1S2
Explicit CoT59.762.062.261.666.367.673.775.477.3
No-CoT27.3–27.235.0–36.539.2–41.0
Coconut28.227.328.636.937.841.245.343.847.0
iCoT32.933.135.236.537.438.742.144.745.6
CODI39.136.225.654.155.448.759.559.762.1
LaDiR20.921.839.323.821.540.622.930.241.5
FLaRe (ours)49.053.656.052.256.660.059.262.965.1

FLaRe leads every latent method on D1 and S2 at every scale, and only lags CODI on D0. Our data also barely helps the other latent methods: from D0 to S2, Coconut, iCoT and CODI gain at most 4.3 points, and CODI even loses up to 13.5, against 6 to 8 points for FLaRe. Even our stage 1 leads the full two-stage LaDiR by 14 to 21 points. Two differences explain why the extra data matters more for a flow, and why the submission compared the baselines under their published setups:

  • Baselines start from the backbone's reasoning. They keep the autoregressive interface their backbone was pretrained for, so they inherit its reasoning ability (Llama-3.2-1B and 3B-Instruct already solve 44% and 78% of GSM8K with 8-shot CoT). Our flow instead learns from scratch to turn noise into codes of a new latent space, and needs 12 epochs to reach 44%.
  • A flow needs both D1 and S2. A flow learns many possible thoughts for each question. Given only one CoT per question, it just memorizes that CoT [1, 2], so Diverse CoT (D1) gives it several to learn from. A flow is also trained on single denoising steps but runs many steps at test time, so its answer pass ends up reading thoughts unlike any it saw in training. stage 2 (S2) trains on these generated thoughts directly, using only questions and final answers. Coconut, iCoT and CODI need neither: they produce one fixed thought per question, and have no train-test discrepancy that requires a stage 2 training.

In summary, with matched data, explicit CoT remains the most accurate method overall, but FLaRe's final model is the most accurate latent method at every scale. Accuracy is not the whole point of a latent thought, so we also probe the thoughts of the retrained baselines (table below). Among latent methods, FLaRe's thought behaves most like an actual chain of thought: its answer follows the thought more faithfully, sampling it covers more answers, and its intermediate values can be read back more often. Its thought can also decode into a full written CoT and improves with more steps. Our claim is therefore the best accuracy among latent methods, with a thought that can be sampled, read and refined, at several times the speed of explicit CoT in the same stack.

1B, matched dataUsefulPass@16 gainValues in order
Coconut, S211.8+3.321.5
CODI, S230.3+4.46.7
FLaRe, S238.3+6.9 (+14.9 decoded)48.7
  • [1] Kadkhodaie et al. Generalization in diffusion models arises from geometry-adaptive harmonic representations. 2024.
  • [2] Gu et al. On memorization in diffusion models. 2025.
  • [3] Yuan et al. Scaling relationship on learning mathematical reasoning with large language models. 2023.
AuthorsOpenReview ↗

General Response 2 - G2: Flare vs. Ladir (part 3)

As the reviewers point out, FLaRe and LaDiR are built from the same components: a variational autoencoder (VAE) over CoTs, flow matching in its latent space, and a shared LLM for both latent thought and answer generation. Neither method introduced them: both reuse ideas of latent diffusion and flow matching for text that predate them [1, 2, 3]. With FLaRe what we add is the training recipe: the paper is a study of how to correctly train a flow-based latent reasoning model. Our argument for the setup we choose are as follows:

  1. Why a flow. We first justify flow matching from the five requirements of latent reasoning. It gives four of them by construction: different noise draws yield different thoughts (diverse), more steps refine them (refinable), the VAE decoder reads them back (explainable), and the thought is a small compressed code whose slots are refined in parallel, so its cost is set by the number of steps rather than the length of the reasoning (efficient). The fifth, useful, depends on training and is the subject of the study.
  2. Why one code. To find which training choices matter, we keep the setup simple: with one latent code for the whole CoT, each choice is measured on its own. With several blocks, its effect would be confounded with effects of the blocks themselves, such as errors carried from one block to the next or when to stop.
  3. How to train it, which is the novelty against LaDiR. Measured one at a time on our model, several of LaDiR's choices are suboptimal: uniform time is the weakest schedule, an answer pass on exact codes loses 6.3 points, and natural language steps lose 9.8 points. In stage 2, LaDiR keeps its supervised rows and only replaces the earlier blocks with its own rollouts, which on our model let the answer bypass the thought. Ours trains on new questions instead: it samples whole thoughts from noise, keeps those that reach the correct answer, and backpropagates the answer loss through the full rollout, so the flow learns to produce thoughts that lead to the correct answer. As a whole, LaDiR's recipe did not transfer well in our experiments: our reimplementation reads 39 to 42 on GSM8K from 0.5B to 3B, against 56 to 65 for FLaRe.

To further show the effectiveness of FLaRe's recipe and disentangle it from the layout of the code, as 6j5k asks, we train both recipes on LaDiR's sequence of blocks as well as on our one code (1B, D1):

LayoutLaDiR's recipeFLaRe's recipeGain
Sequence of blocks (LaDiR)21.540.6+19.1
One code (FLaRe)41.356.6+15.3

As the results clearly show, FLaRe's recipe wins on both layouts: +19.1 points on LaDiR's blocks and +15.3 on our one code, and on LaDiR's blocks our stage 1 alone matches LaDiR's full two stages (40.6). In summary, while FLaRe and LaDiR indeed share their foundations, the paper's novelty is a new and tested recipe for training flow-based latent reasoning models.

  • [1] Lovelace et al. Latent diffusion for language generation. 2023.
  • [2] Zhang et al. PLANNER: Generating diversified paragraph via latent language diffusion model. 2023.
  • [3] Lipman et al. Flow matching for generative modeling. 2023.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.