LDIR: A Local Direction-Informed Riemannian Metric for LLM Reasoning Trajectories

Large language models (LLMs) generate reasoning as trajectories through high-dimensional hidden-state spaces, while these trajectories are commonly analyzed using fixed Euclidean geometry. Such a fixed geometry may fail to capture the nonlinear structure of learned representations and the directional organization of reasoning dynamics. We propose LDIR, a Local Direction-Informed Riemannian framework for analyzing chain-of-thought (CoT) trajectories under a geometry adapted to both representation structure and local reasoning directions. We first map hidden states to a latent space using a geometry-preserving variational autoencoder (VAE) and use its decoder to induce a pullback metric. We then refine this geometry using the local directional statistics of reasoning transitions, yielding a post-pullback anisotropic metric that captures local reasoning dynamics. We evaluate the resulting geometric features on controlled matched valid–invalid reasoning pairs from MATH500 across multiple language models and model families. LDIR consistently provides additional discriminative information over the corresponding pullback geometry induced by the Euclidean metric in hidden-state space, with particularly clear improvements across several Qwen models and additional gains in cross-model settings. Feature-family ablations and trajectory-length controls further indicate that the improvement is not attributable to a single geometric statistic or to reasoning-step count alone. Matched-pair analysis shows that invalid reasoning generally exhibits reduced trajectory movement, lower local speed, and reduced cumulative turning, suggesting that reasoning validity is reflected in the local geometry of hidden-state dynamics. Our code will be made publicly available.