- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FLaRe, a latent reasoning approach based on flow matching. A VAE encodes symbolic chains of thought into compact latent codes, while a shared model learns to generate these codes from a question and use them to predict the final answer. The model is trained to read its own imperfectly generated codes and is further optimized through verified self-training. The authors evaluate the resulting latent thoughts in terms of usefulness, diversity, explainability, refinability, and efficiency.
Critical strengths
- The paper presents a coherent end-to-end recipe for flow-based latent reasoning and evaluates latent thoughts along five dimensions beyond answer accuracy.
- The experimental study is extensive. The authors compare FLaRe with several latent reasoning baselines and report the five proposed diagnostic measures. They also ablate code capacity, VAE corruption, symbolic and dual-route targets, flow time sampling, and the inputs used to train answer generation. These results help identify which choices affect the reported accuracy.
Critical weaknesses
- The paper does not clearly distinguish FLaRe from LaDiR, which already uses a VAE, flow matching, a shared model for latent generation and answer prediction, and rollout-based training. FLaRe's main architectural difference is to generate a single code for the entire CoT rather than a sequence of latent blocks. The comparison in Table 8 changes several representation and training choices together, so it cannot establish how much of the gain comes from this difference. This makes the core novelty and its contribution to performance difficult to assess.
- The link between the five proposed criteria and the training recipe is not established clearly. The ablations in Section 3 mainly report direct and decoded answer accuracy; they do not show whether each training choice improves the property it is intended to support. Thus, the evidence for systematically addressing all five criteria remains incomplete.
Questions for authors
- Why generate an entire reasoning chain in one latent block rather than generate blocks sequentially as in LaDiR? Sequential blocks can naturally accommodate longer and variable-length reasoning, whereas a single code must hold the full chain. Appendix A.4 considers variable-width codes on GSM8K, whose symbolic CoTs are short and already fit in eight slots. Is there evidence that the accuracy and efficiency advantages persist on substantially longer reasoning traces?
- What motivates the selection of these five properties as requirements for effective latent reasoning, rather than as task-dep
5 comments
Response to W1 and Q1 (part 2)
W1. Distinction from LaDiR. We thank the reviewer for this point, and the revision now positions FLaRe clearly against LaDiR. Please refer to the general response (G2) for the shared foundations, the measured choices, and both recipes trained on both layouts.
Q1. Why one code rather than sequential blocks? We thank the reviewer for this question. On GSM8K, one code is both more accurate and faster than a sequence of blocks, and it keeps the study focused on how to train the flow:
- More accurate. With the same recipe, one code beats LaDiR's blocks by 16.0 points with our recipe (56.6 against 40.6) and by 19.8 with LaDiR's (41.3 against 21.5, please refer to G2). Blocks add errors that carry over from one block to the next, and a decision of when to stop. Against LaDiR's full method, the lead of one code holds at every number of reasoning steps, and is largest in relative terms on the longest chains (table below).
- Faster. One code is denoised once for the whole thought, so its sequential passes are set by the number of flow steps. Blocks need a full denoising per block, so their passes grow with the length of the reasoning.
- Focused. With one code, every ablation measures the recipe alone, without the extra choices that blocks bring (block size, when to stop, how each block conditions on the earlier ones), which could confound the takeaways on how to train the flow.
| Reasoning steps | 1 | 2 | 3 | 4 | 5 | 6+ |
|---|---|---|---|---|---|---|
| Questions | 38 | 356 | 394 | 292 | 148 | 91 |
| LaDiR, 1B | 34.2 | 64.3 | 43.1 | 31.2 | 20.9 | 2.2 |
| FLaRe, 1B | 60.5 | 78.1 | 68.0 | 49.3 | 42.6 | 15.4 |
We also note that none of the recipe's choices assumes a fixed width, and Appendix A.4 trains a variable-width code with almost no change, though with no gain on GSM8K since eight slots already give almost 100% exact reconstruction. We discuss much longer traces below.
Q1. Longer reasoning traces. GSM8K chains are short, so we cannot yet show directly that the advantages persist on much longer traces, and the revision scopes its empirical claims to GSM8K-length reasoning. Three results nevertheless point that way:
- Accuracy within GSM8K. On the longest chains the code still reconstructs almost every CoT exactly (93.8% above 40 tokens, table below), so accuracy falls because the questions get harder, not because the code runs out of room. FLaRe's lead over LaDiR is also largest on the longest chains (table above).
- Accuracy beyond GSM8K. A much longer trace would be split into a sequence of codes, generated one after the other by the same flow, as in LaDiR and in latent and block diffusion models of long text [1, 2], rather than held in one wider code, since more slots lower accuracy even on GSM8K (Tab. 1). Our recipe already transfers to this layout without change, and lifts LaDiR's blocks by 19.1 points (G2). None of its choices refers to the length of the trace.
- Efficiency. FLaRe's speed advantage depends on how many tokens each code replaces, not on the length of the trace. Within GSM8K, it spends the same passes on every question, while explicit CoT spends one per token. On longer traces, the advantage persists as long as each segment holds more tokens than the flow takes steps, as on GSM8K, where one or two steps already make FLaRe about 5× faster at 1B.
| Gold CoT tokens | ≤15 | 16–20 | 21–25 | 26–30 | 31–40 | 41+ |
|---|---|---|---|---|---|---|
| Exact reconstruction | 100.0 | 100.0 | 100.0 | 99.1 | 98.6 | 93.8 |
| Direct accuracy | 80.0 | 71.8 | 71.3 | 53.2 | 43.3 | 20.1 |
- [1] Barrault et al. Large concept models: Language modeling in a sentence representation space. 2024.
- [2] Arriola et al. Block diffusion: Interpolating between autoregressive and diffusion language models. 2025.
Response to W2 and Q2 (part 3)
W2 and Q2. The five requirements and the recipe. We thank the reviewer for these questions. The five requirements are not tied to FLaRe: they follow from what any latent method changes with respect to explicit CoT, so they apply to the whole family of latent reasoning methods, ours included. The revision derives them first, then links them to the recipe with probes.
Why these five (Q2). Explicit and latent reasoning both draw a thought and answer from it, . They differ only in what is: written tokens, or a continuous vector that is never verbalized. The requirements are what this change must gain and what it must not lose: it must gain efficiency, the reason to leave the vocabulary, and keep the four properties explicit CoT has by construction:
| Requirement | In explicit CoT | For a latent thought |
|---|---|---|
| Efficient | one pass per token | it costs less at a similar accuracy |
| Useful | the answer reads the written steps | the answer depends on it |
| Diverse | is sampled | it remains a distribution |
| Refinable | a longer chain spends more compute | more compute improves it |
| Explainable | the steps are in language | it decodes to the CoT the answer follows |
Useful is the precondition, since a thought the answer ignores brings none of the others. Since the five follow from the definition of latent reasoning and not from a task, they do not depend on the task, only the probes that measure them are specific to arithmetic.
From the requirements to the recipe (W2).
- Why flow matching. Prior latent methods gain efficiency but lose much of the rest (Sec. 4): their thoughts can be removed or swapped, each question gets one thought, the budget is fixed at training, and nothing reads the thought back. Flow matching in a VAE latent space gives four requirements by design: noise draws give diversity, denoising steps refinement, the decoder explanations, and a small code refined in parallel efficiency. What it does not give is a useful and accurate thought, and this is what our recipe provides (Sec. 3).
- Requirements, not training objectives. We do not optimize them directly, and since their weight depends on the application, we report them separately rather than as one score.
- Each recipe choice, probed. As the reviewer asks, we now run all five probes on the main ablation arms of Sec. 3, not only accuracy:
| 1B arm | Useful | Diverse | Explainable | Refinable | Efficient |
|---|---|---|---|---|---|
| Final stage 1 | 39.7 | +16.6 | 46.0 | +0.4 / +18.2 | 82% |
| Final stage 2 | 38.3 | +14.9 | 48.7 | +2.0 / +23.5 | 85% |
| Exact-code answer pass | 32.0 | +11.9 | 45.0 | −0.5 / +20.2 | 69% |
| Uniform timeᵃ | 22.6 | +24.0 | 39.7 | −1.2 / +2.0 | 47% |
| Single CoT per question (D0) | 36.5 | +11.8 | 43.7 | −0.6 / +17.9 | 79% |
| Single reconstruction route | 31.8 | +12.5 | 42.9 | −1.9 / +9.5 | 75% |
Compared with final stage 1, removing any one of these choices makes the thought less useful, less explainable and less efficient. The two exceptions are on the decoded reading: uniform time samples more widely, from a much lower greedy accuracy (48.8), and the exact-code arm gains slightly more from steps.
Response (part 1)
We thank the reviewer for their detailed review and their suggestions. The revised paper positions FLaRe against LaDiR, shows that our recipe wins on LaDiR's blocks as well as on our one code, scopes the claims to GSM8K-length reasoning while showing how the recipe extends to longer traces, and links each recipe choice to the five requirements with probes.
Below, we answer each weakness (W) and question (Q) in turn, together when they overlap. Two concerns shared with other reviewers are answered once in the general response, whether the gains come from the recipe or from the data (G1) and how FLaRe differs from LaDiR (G2), and we refer to it for W1 and Q1.