Geometric Decomposition of Reasoning Logic in Latent Spaces

Latent chain-of-thought (CoT) reasoning methods train LLMs to reason with continuous latent states, reducing language CoT tokens to a small fixed budget. However, many latent reasoning methods rely solely on *outcome-rewarding objectives* which mainly supervise whether the latent chain reaches an answer state while giving little guidance to the reasoning space itself. In this paper, we study latent reasoning from the perspective of *global reasoning geometry*. Based on compact singular value decomposition, we introduce a **geometric decomposition** of the latent reasoning chain, separating the reasoning space into the principal direction and the orthogonal subspace. We empirically connect the two geometric objects with different aspects of reasoning logic: the principal direction is associated with the core logical form, while the orthogonal subspace is associated with semantic transitional logic in reasoning. Furthermore, we propose a **geometric decomposition-based auxiliary objective** for distillation-based latent reasoning. It aligns the latent reasoning chain with the language CoT's principal direction, and uniformly expands the latent reasoning chain in the orthogonal subspace. We theoretically and empirically show that outcome-rewarding models can lead to ill-conditioned reasoning as intermediate states converge to an "ideal exit" in the reasoning space, which harms the reasoning ability on difficult problems. In contrast, our geometric decomposition-based objective alleviates the latent chain convergence, expands the orthogonal subspace and provides richer reasoning information for answer decoding. Experiments on Llama 3 and Qwen 3 models show 12.7%-35.5% average Pass@1 relative improvements over outcome-rewarding methods on difficult mathematical reasoning settings, validating its effectiveness and geometric interpretability. Code is available at https://anonymous.4open.science/r/gdF451 .