acceptodds
Under review as a conference paper at ICLR 2027

SoftSphere: Understanding and Mitigating Collapse in Latent Reasoning

Abstract

Latent reasoning feeds continuous representations back into a language model, but repeated feedback can reduce their diversity. We study and characterize this representation collapse. Our analysis shows how contracting feedback suppresses trajectory differences and how nondegenerate token sampling can replenish variation. Inspired by these observations, we propose SoftSphere, which projects each soft token onto a sphere whose radius matches the mean vocabulary embedding norm. This correction preserves direction and introduces no trainable parameters or auxiliary loss. We establish guarantees for preserving temporal effective rank and slowing state alignment. We evaluate SoftSphere on both token-anchored and noise-anchored latent reasoning modes across a variety of RL tasks, which show consistent improvements in both final reward and diversity. SoftSphere also induces fewer repetitions in the reasoning chain and greater trajectory effective rank. These findings support controlling geometry as a simple way to improve the usefulness of RL rollouts in latent reasoning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Latent representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PyNuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies representation collapse in soft-token latent reasoning, distinguishing temporal covariance saturation, attenuation of input perturbations, and loss of diversity across rollouts. It proposes SoftSphere, which projects each soft token onto a sphere whose radius matches the mean vocabulary-embedding norm. Conditional theoretical results establish preservation of temporal effective rank and slower state alignment. Experiments across two model families and five tasks show improved temporal rank, reduced repetition.

Critical strengths
  1. The paper clearly distinguishes variation within a trajectory, local input sensitivity, and diversity across sampled trajectories. It also explicitly recognizes that covariance shrinkage does not necessarily reduce effective rank. These distinctions make the analysis of latent collapse more precise and provide a useful diagnostic framework for continuous feedback.

  2. SoftSphere is a simple intervention requiring no additional parameters or auxiliary objective.

Critical weaknesses
  1. The contribution beyond standard normalization remains insufficiently established. Fixed-norm normalization has clear precedents [1,2]. Although the paper studies partial and centered variants, it does not isolate the benefit of per-step norm correction from a constant rescale. Its comparative value relative to collapse-mitigation methods such as WIDER [3] and SeLaR [4] also remains unclear.

  2. The cross-rollout diversity claim lacks direct evidence. Cross-rollout collapse (Fig. 2c) is an important motivation, but SoftSphere’s effect on this collapse is not shown. The reported temporal-rank and pass@k improvements do not by themselves establish greater diversity across rollouts.

[1] Zhang & Sennrich. Root Mean Square Layer Normalization. NeurIPS 2019.

[2] Loshchilov et al. nGPT: Normalized Transformer with Representation Learning on the Hypersphere. ICLR 2025.

[3] Hao & Yang. Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning. 2026.

[4] Fu & Luo. SeLaR: Selective Latent Reasoning in Large Language Models. ACL 2026.

Questions for authors
  1. What evidence distinguishes the benefit of per-step norm correction from ordinary scale adjustment, and what does SoftSphere contribute beyond established normalization and collapse-mitigation methods [1–4]?

  2. Which reported result directly supports mitigation of cross-rollout collapse, rather than improved within-trajectory temporal rank or answer coverage?

  3. Which observed behavior in the actual unprojected trajectories corresponds to the sustained radial decay assumed by Theorem A, given the reported mean norm of approximately 0.99R?

Reviewer_HWdoOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies "representation collapse" (temporal covariance rank saturation within a rollout, fading sensitivity to small input perturbations, and loss of diversity across rollouts) in soft-token latent reasoning and propose projects each soft token onto a fixed-radius sphere (SoftSphere). The paper supports it with theoretical analysis on effective rank and alignment, and with experiments on HRPO and SofT-GRPO using Qwen2.5-1.5B and Llama-3.2-1B across five tasks. Modest gains, mainly at pass@8.

Critical strengths
  • Simple, plug-in method. It adds no parameters or auxiliary loss, and the same change applies to two different latent-reasoning methods.

  • Many experiments with acceptable results. The evaluation covers two model families and five tasks, pass@8 and pass@16 gains are good. I appreciate that the authors report the cases where the method does not help and acknowledge the limits. No method is perfect, and showing negative results openly will help follow-up work.

  • Clear figures and tables. The figures and tables are well designed and easy to read.

Critical weaknesses
  1. Motivation is problematic

    • The key intuition is "if each pass dampens differences between states, feeding its output back repeatedly compounds that damping". No justification or reference is given, and I don't think a Transformer forward pass does this. Fig. 2b actually shows the opposite: a single pass can amplify small differences (×6.08).
    • Even ignoring that, I can't see why this motivates SoftSphere. Paragraph 3 says the lost variation can be restored by "stochastic latent methods can supply it through noise". So why do we need a sphere? Appendix D even says the sampling effect is "the component that a deterministic projection cannot supply".
    • Paragraph 4 switches to a different premise: "when successive states shrink, later directions contribute less to the trajectory's variation". This corresponds to an assumption in Theorem A, but the paper never proof such assumption.
  2. Theories are based on erroneous assumptions

    • Theorem A assumes radial decay, direction dynamics independent of radius.. Neither is checked, and the paper's own data contradict the first: the projection gain is only 1.037 / 1.025, and the unprojected ‖s‖/R is already 0.99 in Table 7.
    • Theorem B is a toy setting. It simplifies the latent feedback loop to $F(h) = a,h + (1-a),h^\star $.. This is far from how latent reasoning (transformers with softmax-weighted embeddings as input, definetly and highly nonlinear). Moreover, the authors did not even explain clearly what this theorem B shows other than a clear defendive conclusion "The guarantees concern within-rollout geometry, however, and do not imply diversity across generations or higher pass@k."
  3. Terminology and notation are sloppy

    • "Terminal-reward RL" is never defined.

    • "Feedback" is used with no definition. In RL it means the reward signal, but here it seems to mean feeding outputs back as inputs.

    • Three different collapses (temporal rank, input sensitivity, ensemble rank) are all called "collapse" or "rank." I cannot tell which result supports which claim, especially when figure 1, 3 and 4 are not even referenced in main content.

    • The state is written as sts_t, hth_t, and ztz_t in different places, and the paper never says whether they are the same object.

    • Many terms are used before being defined or are used with several meanings: HRPO and SofT-GRPO are never expanded; "pure-soft," "state alignment," "radial contraction," and "angular diversity" are used without definition; and "pass," "feedback," aa, RR, KK, and "Theorem A" each denote different things in different places.

Overall I can see quite a lot of inconsistencies and contradictions in the paper's motivation, theory, and experiments. The writing logic, wording and terminology makes it very difficult to understand and follow. I don't think such manuscript is ready for acceptance.

Questions for authors

See weaknesses.

Reviewer_C3woOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SoftSphere projects a soft latent token onto a sphere whose radius is set by the average vocabulary-embedding norm. The paper distinguishes temporal effective-rank collapse, alignment toward a fixed point, and diversity across generated solutions, and gives conditional analyses of the first two phenomena. The projection is evaluated within HRPO and SofT-GRPO using two small language models and several reasoning tasks. It improves some geometric diagnostics and several best-of-n results, although its effects vary considerably across models, tasks, and rollout mechanisms.

Critical strengths
  1. The proposed intervention is simple and does not introduce additional learned parameters. This makes it appealing as a practical modification to latent reasoning methods.
  2. The distinction between temporal rank collapse, fixed-point alignment, and diversity across solutions is useful and helps clarify an otherwise ambiguous discussion of collapse.
  3. The evaluation includes two rollout mechanisms, several tasks, and normalization ablations. The paired comparisons make it easier to see where the intervention helps and where it does not.
Critical weaknesses
  1. The theoretical explanation is not fully convincing for the setting studied. The analysis relies on restrictive assumptions and simplified dynamics, while it is unclear how closely the actual rollouts follow these conditions. The connection between the theoretical claims and the empirical behavior remains loose.
  2. The methodological contribution appears modest. The main intervention is a normalization step, and the current evidence does not clearly separate a collapse-specific benefit from the more general effect of changing the scale of the model inputs. The normalization ablations are useful, but do not fully resolve this concern.
  3. The practical gains are inconsistent across tasks and models, with several regressions. I also found the aggregate results harder to assess than they should be, since the run-level breakdown and training-seed variability are not sufficiently clear. This makes the overall claim of a reliable improvement premature.
Questions for authors
  1. Can the authors clarify the runs included in the aggregate HRPO results and report the variability across training seeds?
  2. To what extent do the measured rollouts satisfy the assumptions used in the theoretical analysis?
  3. How sensitive are the improvements to the chosen radius, and can they be separated from a general input-scaling effect?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.