- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces LLoCoT. LLoCoT abandons standard left-to-right latent thought generation and instead refines a fixed set of K=64 learned slot vectors. A shared transformer, the full Qwen3 1.7B Base backbone, runs R=2 times over the slots and prompt; a diagonal Gaussian head outputs K 2560-dimensional latent thoughts, and predicted means from non-final loops feed back into hidden states. Final latents are sampled in parallel to drive an autoregressive answer decoder. Training targets follow NF-CoT: a standalone transformer VAE trained for 180k steps encodes reasoning traces into 64 continuous vectors, then a shallow conditional autoregressive flow maps these vectors into the model’s latent space. The loss combines likelihood over final latents (including the flow’s log determinant), an intermediate prediction auxiliary likelihood weighted at 0.25, and answer cross-entropy for the decoder conditioned on target latents. Training uses two phases: roughly 1.5k steps with frozen backbone, then around 45k full fine-tuning steps at batch size 64. Evaluated on HumanEval, HumanEval+, MBPP and MBPP+ with 16 samples per task, its average score across the four benchmarks hits 63.6, versus 63.2 for Reasoning SFT, 62.7 for answer-only SFT, 62.0 for NF-CoT and 57.3 for the base model. Tested at batch size 1 on a single H100, time to first answer token falls from 3.17 s (Reasoning SFT) to 0.09 s and end-to-end latency from 15.16 s to 11.47 s, though peak memory rises 1.9× and analytical FLOPs go up 1.6×.
Critical strengths
The efficiency measurement is carefully done and honestly described for what it shows. The serial reasoning phase falls from about 2.5 s (Reasoning SFT) and about 2.0 s (NF-CoT) to 0.06 s, with a clear protocol (128 prompts, three runs after ten warm ups, batch size one, phase by phase timings) and forward call counts that explain the gain. The authors say plainly that memory and FLOPs are worse and that the speedup comes from parallelism rather than less compute. That is the right framing, and the idea of replacing serial latent generation with a small number of parallel refinement steps is a sensible direction.
Second, the evaluation protocol is unusually clean for this area. Appendix B states that failed, truncated and malformed completions count as zero rather than being dropped, that every task by sample record must exist before results are promoted, and that pass@1 is averaged over 16 samples per task. The paper is also candid in its limitations section (one 1.7B model, code only, no adaptive depth).
Critical weaknesses
the accuracy claim is not supported at the level the abstract implies. LLoCoT's four benchmark mean (63.6) is 0.4 points above Reasoning SFT (63.2) and 0.9 above answer only SFT (62.7). It is behind Reasoning SFT on HumanEval (69.3 vs 70.3) and HumanEval+ (63.9 vs 64.4) and ahead only on MBPP and MBPP+. The intervals the paper reports are fixed task sampling intervals (half widths of 0.6 to 1.3 per benchmark), which by the paper's own words (Appendix B.3) do not capture variation over training seeds, and each system appears to be a single training run. The ablation conclusions are even smaller than that: final configuration vs no feedback is +0.5, auxiliary loss alone is minus 0.2, sampled feedback is minus 0.7, and the loop depth sweep moves the mean by 0.1 to 0.6 for R=1 to 4. These differences are well inside plausible seed to seed variation for a 1.7B model, so the statements about which feedback or loop setting is best are not established. The honest claim the data support is "on par with Reasoning SFT in accuracy".
Questions for authors
1/ What data were used to train the VAE and the LLoCoT stages (problems, reasoning trace source, size), and what exactly is "Reasoning SFT"?
2/How many training seeds underlie each row in Table 1 and what is the seed to seed spread? Do Base SFT, Reasoning SFT and NF-CoT receive the same number of optimizer steps and examples as LLoCoT?