- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Autoregressive Thought Flow (ATF) for continuous latent reasoning. A causal Transformer provides the context, and a lightweight flow head samples the next latent thought. Each sampled thought is fed back into the model. A frozen Causal-Register VAE provides training targets. Experiments show accuracy gains over a matched regression baseline and broader solution coverage through latent sampling.
Critical strengths
-
ATF performs repeated denoising in a small flow head rather than repeatedly running the full backbone. The lightweight generation design has practical value.
-
The matched MSE baseline shares the backbone, latent targets, and stopping rule with ATF. Greedy answer-decoding experiments also show that latent sampling alone can produce useful alternative answers. These results support generative next-thought prediction beyond text sampling. ATF improves the accuracy–latent-length trade-off in several settings.
Critical weaknesses
-
The additional benefit of learning a flow generator is not fully isolated. The matched MSE comparison does not separate multimodal expressiveness from stochasticity, leaving unclear whether simpler dropout or noise-based sampling could provide similar gains. The LaDiR comparison also involves different training and RL procedures. Moreover, the mixed majority-vote results against MSE do not establish a consistent advantage in final-answer selection.
-
The interpretation of the learned latent modes is not sufficiently established. The paper motivates its approach through multiple plausible next thoughts, but the evidence connecting latent modes to distinct, valid reasoning continuations is mainly qualitative. The PCA plots and decoded examples illustrate variation, but do not establish how consistently this variation reflects meaningful reasoning alternatives.
Questions for authors
-
Does the CoLaR comparison isolate the prediction head? Under matched training and inference budgets, what additional benefit does learning a flow generator provide over adding dropout or Gaussian noise to the matched regression baseline? What evidence links this benefit to distinct, valid reasoning modes?
-
Under the same answer-selection rule and total inference budget, do the Pass@k gains translate into higher accuracy for the selected answer?
-
How are the discrete CoT budgets in Table 1 enforced, particularly the transition from reasoning to answer generation? Why does Qwen3-4B achieve 72.18% on GSM8K with a 500-token budget but only 47.84% with a 1,000-token budget? (I am mainly curious about the reason for this result, and this is not a criticism of the core method.)
4 comments
Q1. CoLaR Head Isolation and Flow vs. Dropout / Gaussian Noise on Matched MSE
CoLaR (Table 3) compares complete systems under a shared backbone and dataset rather than isolating the prediction head alone, as CoLaR uses its own pooled-embedding targets and training recipe rather than our frozen CR-VAE.
To strictly isolate the prediction head, we evaluate additive Gaussian noise and MC-dropout ( and , following You et al., 2026) on our matched Qwen3-4B MSE baseline (, ):
| Method (Qwen3-4B, SFT-only, no RL) | MATH-500 | OlympiadBench | ||
|---|---|---|---|---|
| Pass@1 | Pass@16 | Pass@1 | Pass@16 | |
| Deterministic MSE | 77.0 | 87.8 | 40.8 | 58.7 |
| MSE + Gaussian Noise | 77.5 | 90.0 | 40.4 | 59.6 |
| MSE + MC-Dropout (p=0.03) | 76.5 | 90.8 | 39.9 | 60.8 |
| MSE + MC-Dropout (p=0.20) | 66.1 | 93.2 | 31.2 | 59.2 |
| ATF (Flow Head) | 78.7 | 92.2 | 41.9 | 62.5 |
Compared with adding MC-dropout or Gaussian noise to matched MSE, learning a Flow Head provides two key benefits:
- Mutually reinforcing per-path correctness (Pass@1) and multi-sample exploration (Pass@16): Over deterministic MSE (77.0 / 40.8 Pass@1; 87.8 / 58.7 Pass@16), ATF simultaneously achieves the highest Pass@1 (78.7 / 41.9) and significantly raises Pass@16 (92.2 / 62.5; at greedy , MATH-500 Pass@1 → Pass@16 rises 76.8 → 88.8 vs. flat 76.8 for MSE). By contrast, Gaussian noise and mild dropout () under-explore (90.0 / 59.6 and 90.8 / 60.8), while strong dropout () raises MATH-500 coverage only at a 10.7–14.5 point drop in Pass@1 (and still trails ATF on OlympiadBench Pass@16: 59.2 vs. 62.5).
- Providing a learnable multimodal policy for RL: MC-dropout lacks a tractable trajectory likelihood, and Gaussian noise adds only fixed, unlearnable isotropic variance around a single MSE prediction. ATF's Flow Head instead parameterizes a trainable conditional policy over next thoughts that latent-trajectory RL directly optimizes, lifting 8B Pass@1 by +5.0 to +12.5 points (W1) and more than doubling answer-only RL's gain on 1B GSM8K (+3.8 vs. +1.8).
Link to distinct, valid reasoning modes. While strong dropout () can raise MATH-500 Pass@16 at a steep cost to single-sample accuracy, ATF achieves strong Pass@16 alongside the highest per-path Pass@1 (78.7 / 41.9), confirming valid paths, and the greedy-decoding analysis in W2 shows that these trajectories yield genuinely distinct mathematical approaches (33.8% vs. 26.0% for the Gaussian noise baseline).
[Q2, W1] Selected-Answer Accuracy and Comparison with MSE Under Matched Budget
Yes, the Pass@ gains consistently translate into higher selected-answer accuracy. We adopt majority voting (Maj@16) as the shared answer-selection rule, and to isolate the contribution of the Flow generator without RL confounds as pointed out in W1 (extending Table 11), we compare Qwen3-4B (SFT-only, ) against matched deterministic MSE across the three large Table 1 benchmarks (GSM8K, MATH-500, OlympiadBench), reporting Maj@16 alongside sequential 16-rollout inference time (Time (s), following Appendix E Table 9 on 1 RTX PRO 6000 at batch size 1):
| Method | GSM8K | MATH-500 | OlympiadBench | |||
|---|---|---|---|---|---|---|
| Maj@16 | Time (s) | Maj@16 | Time (s) | Maj@16 | Time (s) | |
| Deterministic MSE | 91.0 | 62.1 | 79.8 | 113.7 | 44.9 | 128.4 |
| ATF (Flow Head) | 91.6 | 63.9 | 82.4 | 121.8 | 47.2 | 139.4 |
- Consistent selected-answer gains over deterministic MSE: ATF achieves higher Maj@16 across all three benchmarks.
- Strictly smaller or matched total inference budget: Of the 1.8–11.0 s total difference across 16 rollouts, the 24 Flow Head steps add s ( of runtime), with the rest coming from 3–46 slightly longer answer tokens per rollout. Moreover, even at half the sampling budget (Maj@8: 32.0 / 60.9 / 69.7 s, of MSE's 16-rollout time), ATF reaches 91.6 / 82.1 / 46.4, already surpassing MSE's Maj@16 across all three benchmarks.
Q3. Discrete CoT Budget Enforcement in Table 1 (500 vs. 1,000 Tokens on GSM8K)
We thank the reviewer for pointing out an insightful detail that helps us refine our budgeted-CoT evaluation protocol! For capped discrete CoT (, ), our evaluator runs two phases: Phase 1 generates up to the thinking budget (appending </think> to transition to the answer only if </think> has not yet appeared), and Phase 2 generates up to 1,024 final-answer tokens.
Upon inspection, we found that when a problem is simple enough for the model to naturally finish both thinking and its final answer (EOS) within Phase 1, Phase 2 still continued generating past EOS, appending extra text whose trailing \boxed{} could overwrite the already-completed answer. Because GSM8K requires much shorter reasoning traces than the harder competition benchmarks (804 thinking tokens unbounded at 4B vs. 2,020–3,721 elsewhere), raising the cap from 500 to 1,000 caused most 4B GSM8K problems (958 / 1,319, vs. 136 at ) to finish inside Phase 1, producing this reversal on GSM8K. Stopping cleanly at each item's first EOS restores the expected monotonic trend on 4B GSM8K (79.15% at → 91.74% at , matching unbounded 91.21%). We will update Table 1 with this refined protocol in the revision, and all conclusions regarding ATF remain unchanged.
Thank you for the additional experiments, which help alleviate my main concerns. I have increased my score accordingly. I would encourage the authors to add majority-vote results for the stochastic MSE baselines under the same inference budget, and to update the paper with the corrected CoT results and revise the related conclusions as appropriate.
We thank Reviewer Rht7 for the thoughtful review and for recognizing our lightweight Flow Head and matched MSE design. Below we address all concerns via additional experiments on stochastic MSE baselines, latent modes, and matched-budget selection, plus clarifications on CoLaR/LaDiR and Table 1 CoT budgets.
W1. Demonstrating the Isolated Benefits of Learning a Flow Generator
Additional matched noise-based MSE comparisons. We thank the reviewer for suggesting the MC-dropout and Gaussian-noise baselines. Controlled experiments on our SFT-only matched MSE model (Q1 below) highlight two advantages of ATF: (1) mutually reinforcing per-path correctness (Pass@1) and multi-sample exploration (Pass@16) with no test-time tuning, whereas both MSE perturbations face an accuracy–exploration trade-off; and (2) providing a learnable multimodal policy, whereas neither heuristic perturbation defines a trainable distribution, making extension to latent RL difficult.
Aligned comparison with LaDiR. For a direct RL-free comparison in Table 2 (LLaMA-3.1-8B), we report Pass@100 for SFT-only ATF (Flow SFT) alongside LaDiR:
Before any RL, Flow SFT already surpasses both LaDiR w/o Stage 2 and full LaDiR (as well as all other Table 2 baselines) in Pass@100, while RL subsequently concentrates probability onto correct paths to raise Pass@1.
Expanded majority-vote comparison against MSE. Beyond Table 11, evaluating majority voting across the three Table 1 benchmarks (GSM8K, MATH-500, OlympiadBench) on Qwen3-4B ([Q2, W1] below) shows that ATF consistently outperforms matched deterministic MSE in Maj@16.
W2. Quantitative Analysis of Learned Latent Modes
To move beyond qualitative examples and isolate variation arising purely from continuous latent thoughts, we evaluate Qwen3-4B across all 500 MATH-500 problems under greedy answer decoding (T=0, K=16), comparing ATF against deterministic MSE and MSE + Gaussian Noise. Beyond exact-string diversity, an LLM judge (Qwen3-30B-A3B-Instruct, T=0) groups each problem's correct rollouts by underlying mathematical strategy (ignoring wording or reordered arithmetic; counted conservatively only when two randomized presentation orders agree, 90.6% agreement):
With text-decoding randomness turned off (T=0, where deterministic MSE yields a single trajectory per problem), ATF consistently translates latent variation into distinct valid reasoning paths across the dataset: it produces ≥2 distinct correct solutions on 83.0% of problems (+12.8 over Gaussian noise, paired 95% CI excluding 0) and ≥2 substantively different mathematical approaches on 33.8% of all problems (+7.8 over Gaussian noise, paired 95% CI [+2.6, +13.0]). We will add this dataset-level quantitative analysis to Section 4 in the revision.