- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Selective Latent Thinking (SLT), a reasoning paradigm that adaptively interleaves implicit latent representations with explicit CoT within a single reasoning trajectory, addressing the high inference cost of explicit chain-of-thought reasoning and the problem that existing latent reasoning methods adopt a "homogeneous compressibility assumption," causing precision-critical spans to be over-compressed. To this end, the authors introduce a lightweight feature decoder that anticipates short-horizon future reasoning trajectories, and a confidence gate that selects the longest continuous prefix that can be reliably compressed; this prefix is encoded by a cross-attention compressor into a fixed-length latent block, while the remaining precision-critical spans are preserved as explicit CoT. The framework is trained in three stages (latent compression learning, reliability-aware feature decoding, and trajectory-level GRPO optimization), with a dual-ascent constraint on the latent share to prevent the model from retreating to fully explicit reasoning. The discussion of the deficiencies of the homogeneous compressibility assumption is generally clear, and the design of the compression and reliability mechanisms is reasonably well motivated. The main contribution lies in shifting the "what to compress" decision from a uniform assumption to span-wise decisions based on per-span reliability, trading off answer correctness against explicit token cost via trajectory-level reinforcement learning, and transferring this principle to continuous-thought models (Latent-SLT). The authors conduct systematic experiments and ablations on mathematical reasoning benchmarks such as GSM8k-AngNL: explicit reasoning length is reduced by 58.4% with only a 2.8-point accuracy drop on Llama-3.2-1B, and SLT surpasses RoT by 17.33% at a comparable compression ratio on Qwen3-4B; ablations show that random compression drops GSM-Hard accuracy from 54.76% to 17.3%, confirming the decisive role of which spans are compressed. Finally, comparisons against Latent-SFT, Latent-GRPO, and other methods, together with general-ability retention results (MMLU, ARC, HellaSwag), are used to argue for SLT's superiority and generality.
Overall, this paper could be a significant algorithmic contribution, but several theoretical and experimental details require clarification: there is an objective mismatch between the reliability losses and the "longest continuous reliable prefix" rule at inference time (with continuity never explicitly optimized), the rationale for the hyperparameter ρ = 0.73 is not provided, and comparisons against more established non-uniform/adaptive compression baselines are missing. Given clarifications on these points in an author response, I would be willing to increase the score.
Critical strengths
-
The paper designs a dynamic CoT compression method that compresses only those spans amenable to predictable segment-level compression, thereby preserving reasoning accuracy as much as possible. This further confirms that the homogeneous compressibility assumption is fundamentally flawed.
-
The framework is technically complete, with each component supported by ablations, and the empirical results are compelling. This demonstrates that the method is not only conceptually interesting but also practically effective and controllable — the accuracy-length trade-off can be freely adjusted at inference time, which is precisely what determines whether such a method can be adopted in practice.
-
Control experiments (e.g., the acc-only control in Table 8, where the latent share collapses to 16.9% and the length doubles) also show that the authors understand and have addressed the failure modes of their own objective function, which increases my confidence in the experimental conclusions.
Critical weaknesses
-
At inference time, the compression decision follows the rule "take the longest continuous reliable prefix satisfying c_j ≥ τ," but both L_eosch and L_gate on the training side are per-position objectives. No term optimizes the length of the continuous reliable prefix or the temporal continuity of reliability. Consequently, the model may well learn a "jagged" reliability pattern in which position j is reliable, j+1 is unreliable, and j+2 is reliable again — and the prefix rule would discard all reliable spans after j along with it, unintentionally suppressing the compression rate. The paper's response is to explicitly defer long-horizon consequences to Stage 3 trajectory-level RL ("the first two stages establish local mechanisms... but they do not directly optimize the long-horizon consequences"), but the RL reward is merely an ex-post correction (and it only penalizes explicit tokens without constraining prefix structure), not a structural guarantee at the level of the training objective. This is critical, because it undermines the paper's central selling point — that "selective compression decisions are learned through a reliability-aware mechanism"; the disconnect between the decision mechanism and the training signal means that the observed gains may stem in part from RL's global redistribution rather than from reliability estimation itself. In addition, key design details such as the span selection rule in Stage 1 of the main framework and the specific value of the latent block length N are left unspecified, further complicating mechanistic attribution.
-
The experimental baselines consist mainly of pure explicit CoT (SFT-CoT, SFT-w/o CoT), token-level reduction (TokenSkip), and uniform/fully implicit methods (CoLaR, RoT, Coconut, Latent-SFT/GRPO). The problem is this: the paper's core claim is that "selectivity" is what brings the gains, yet apart from two heuristic ablations (Compress-Random, Compress-SkipNum) and one learned-gate control (Ours-RL-gateω), no existing non-uniform/adaptive compression method is included as a baseline. This leaves one alternative explanation unexcluded: the gains may come from "any form of non-uniform compression" (e.g., entropy-based, semantic-boundary-based, or sentence/step-level adaptive compression) rather than from the specific "reliability-aware gating" mechanism proposed in this paper. Meanwhile, although the related work section discusses hybrid ideas such as Coconut, Medusa, and multi-token prediction, no hybrid explicit-implicit reasoning method is directly compared against in the experiments. This point is decisive for my overall assessment.
Questions for authors
-
Inference uses the "longest continuous reliable prefix satisfying c_j ≥ τ" rule: please clarify: (a) how the trailing reliable spans abandoned after prefix truncation are handled at inference time (are they re-considered for compression at subsequent steps)? (b) Have you observed the actual impact of jagged reliability distributions on the compression rate? (c) Have you considered structured objectives on prefix length or continuity?
-
Rationale for key design parameters: (a) How was ρ = 0.73 determined? (b) What is the specific value of the latent block length N? (c) Is the span segmentation rule for constructing hybrid sequences in Stage 1 based on random sampling or some other criterion?
-
Stage 3 RL globally reshapes the compression distribution. Please provide an end-to-end breakdown of compression rate and accuracy using only the Stage 2 gate (without RL), to disentangle how much of the gain comes from "reliability estimation" itself versus trajectory-level RL redistribution.
-
Stage 3 only penalizes explicit tokens (β = 0.01) and does not penalize the number of latent blocks. Under acc-only rewards or extreme thresholds, compression collapse and doubled length have been observed. Please describe under what conditions the method may degenerate into being "longer than SFT-CoT," and whether there are diagnostic metrics to help practitioners avoid this situation.