acceptodds
Under review as a conference paper at ICLR 2027

Rational Manifolds: The Spatial Organisation and Control of LLM Reasoning

Abstract

The thinking traces of large language models (LLMs) interleave behaviours such as planning, exploring and verifying, yet little is known about how these behaviours are arranged in latent space. Linear representations provide a framework to explain and control model behaviour but the directions it relies on are often unstable. We provide a mechanistic recount of LLM reasoning by interpreting regions instead of directions as the semantic units of latent space, partitioning activations via a Mixture of Factor Analyzers (MFA). Our experiments reveal that these regions correspond to recognisable problem-solving behaviours with complex nonlinear boundaries and whose local concept directions consistently disagree with global ones. Causal interventions show that regions offer limited control over individual behaviours but can have a strong impact on reasoning performance: diverting traces away from a small subset of load-bearing regions critically degrades accuracy, but carefully redirecting the path a model takes through specific regions can also make reasoning shorter and more accurate. This mechanism is found to differ geometrically from linear steering. Together, these findings suggest that localised descriptors, such as latent space regions and the paths through them, can inform new methods for both interpretation and control of reasoning.

Primary area:
interpretability and explainable AI
Keywords:
NLP, LLMs, Reasoning, Mechanistic Interpretability, CoT

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7bbAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a region-based view of LLM reasoning. Instead of modeling reasoning behaviors with a single global linear direction, the authors partition the latent space into local regions using a Mixture of Factor Analyzers (MFA) and represent each reasoning trace as a trajectory through these regions. Across multiple reasoning models and benchmarks, they find that different reasoning behaviors tend to occupy different regions, while local behavior directions can differ substantially from global ones. Causal interventions further show that some regions are critical for correct reasoning, and promoting selected regions can improve accuracy and reduce reasoning length.

Critical strengths
  1. The paper challenges the common assumption that complex reasoning behaviors are represented by a single global linear direction, and provides evidence that local regions and local directions better capture the geometry of reasoning activations. This is a conceptual contribution to mechanistic interpretability.

  2. Causal interventions show that region traversal matters for reasoning performance. The paper goes beyond correlation by rerouting and promoting latent regions, identifying load-bearing regions whose removal substantially hurts accuracy and beneficial regions whose promotion can improve accuracy and reduce reasoning length. This strengthens the practical relevance of the proposed framework.

Critical weaknesses
  1. The evidence does not clearly isolate behavior-specific local structure. Figure 3 compares different classification settings, and most gains against the global rest persist after shuffling episode labels within regions. It is therefore unclear whether higher AUROC reflects better behavioral separation or simply region membership.

  2. The reported performance gains are difficult to interpret because some baselines are very weak under the chosen inference setting. The experiments impose a 2048-token thinking budget, and, e.g., Qwen3-32B achieves only 25.7% accuracy on MATH before intervention. The evaluation leaves the source of performance gains and the reliability of baseline comparisons unclear.

Questions for authors
  1. Can global and local directions in Figure 3 be evaluated on identical held-out examples to separate behavior-specific discrimination from region membership?

  2. Why are some baseline accuracies substantially lower than commonly reported numbers for the same models, e.g., Qwen3-32B on MATH and GPQA? Since all experiments use a 2048-token thinking budget, could the authors report whether the intervention gains persist under a larger reasoning budget?

  3. Given the limited behavioral specificity of interventions, what operational meaning is assigned to “semantic units,” and what evidence supports this interpretation beyond statistical association?

Reviewer_7jLzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper interprets reasoning traces as walks over latent-space regions, found by fitting an MFA on sentence-level activations, instead of using global linear directions. It reports four findings:

  1. Episode categories cluster in regions whose local directions disagree with global ones.
  2. Steering toward regions mostly triggers “monitoring” rather than the targeted behavior.
  3. Rerouting a few load-bearing regions collapses on accuracy.
  4. Enhancing selected regions results in shorter trajectories and improved accuracy.
Critical strengths
  1. The region-based view of reasoning and the local-vs-global direction analysis (Figs. 4, 11) is interesting and provides a novel viewpoint and reference for the debate on linear representations.
  2. The figures are clearly visualized and make the geometric claims easy to understand.
  3. Experiments and ablations are comprehensive. The appendix makes ablations on K, layer depth, task and promotion probability, and includes properly corrected statistical tests.
Critical weaknesses
  1. The main conclusion is unclear. The findings point in different directions, and much of the key evidence sits in the appendix.
  2. Novelty is overstated. Hierarchical Thinking steers latent state transitions at sentence boundaries and improves accuracy on similar benchmarks. SOPHIA and AlgoTrace take similar approaches. The paper should discuss and compare against them.
  3. The token budget may be critical to the accuracy claims. Under the 2048-token budget, most traces seem truncated (for example, Qwen3-32B: 1951 ± 6 tokens, 25.7% on MATH), and gains closely track length reductions. Please report truncation rates and evaluate under a realistic budget. For reference, the Qwen3 technical report gives 97.2% on MATH-500 for Qwen3-32B in thinking mode (32k output budget) and 88.6% even with thinking disabled. The 25.7% reported here therefore strongly suggests that the 2048-token budget, not the model, is the limiting factor.
  4. The models are dated. Most models are from 2025 and the largest is 32B, which raises doubts about generalization to current models.
  5. The claim of a “different mechanism” is not supported. In Fig. 10, the axis is mislabeled and there is no random baseline; medians of roughly 0.1–0.3 are well above chance in high dimensions.
Questions for authors
  1. How does this work relate to state-transition steering methods, such as Hierarchical Thinking, SOPHIA and AlgoTrace?
  2. How are truncated traces scored? Are traces that hit the 2048-token budget forced to produce an answer, or are they counted as incorrect? What fraction of traces are truncated for each model and dataset? Do the gains exsit on problems that the baseline solves within budget?
  3. Do the selected regions just trigger early conclusion? Which episode categories do the ACC- and JNT-selected regions belong to? If they are mostly answer-type regions, the gains may reflect the model being pushed to wrap up earlier rather than to reason better.
  4. Why is Gemma-4-E2B unaffected? Have you tried intervening at other layers for this model? This could be helpful if the authors add experiments to explore whether the per-layer-embedding explanation holds or whether region effects depend on architecture more generally.
Reviewer_YJBXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the spatial organization and controllability of reasoning in LLMs. It fits a Mixture of Factor Analyzers to sentence-level hidden representations and describes reasoning traces as walks between latent-space regions. Across eight models and three benchmarks, the authors investigate how these regions relate to problem-solving behaviors and introduce rerouting and promotion interventions. The results show that local behavioral directions differ from global ones, that interventions on a small subset of regions can substantially degrade accuracy, and that selected interventions can improve accuracy and reduce generation length. However, region attribution does not translate into reliable control of individual reasoning behaviors.

Critical strengths
  1. A useful local perspective on reasoning representations. The paper extends region-based representation analysis to reasoning trajectories and provides evidence across multiple model families and tasks that local behavioral directions can disagree substantially with global directions. This offers a useful qualification to approaches that describe complex reasoning behaviors using a single global steering vector.

  2. Interventional evidence beyond descriptive clustering. The promotion and rerouting experiments connect latent-space regions to downstream task performance, with beneficial regions selected on validation data and evaluated on held-out tests. The finding that behavioral attribution is easier than behavioral control is also informative: the paper explicitly reports that targeting behavior-associated regions generally fails to induce the corresponding behavior reliably.

Critical weaknesses
  1. The mechanistic interpretation remains underdetermined. MFA identifies statistical structure in activations, but the evidence does not establish that its components are functional units of reasoning. Regions contain multiple behavior categories, much of the local discrimination advantage survives within-region label permutation, and targeted interventions generally fail to elicit the associated behaviors. Similarly, disagreement between local and global directions supports limitations of a global linear description, but does not by itself establish a strongly curved reasoning manifold. The load-bearing interpretation would also benefit from controls matching intervention locations and perturbation magnitudes, since performance degradation could partly reflect sensitivity to off-distribution perturbations.

  2. The fixed generation budget limits the interpretation of performance gains. All main evaluations use a 2048-token budget, while several baseline generation lengths approach this limit and many GPQA results are below chance. Consequently, improvements may partly arise from helping models complete their responses within the budget. This remains practically useful, but makes it difficult to assess how much the interventions improve reasoning beyond budget management. Results under larger budgets, together with truncation and completion rates, are needed to establish the scope of the gains. Token reductions also do not establish practical speedups without accounting for online MFA inference and reconstruction overhead.

Questions for authors
  1. Do the accuracy improvements persist under substantially larger generation budgets? Please report truncation or completion rates alongside accuracy, particularly for MATH and GPQA, to clarify the contribution of budget-induced failures.

  2. Does the load-bearing effect remain stronger than random perturbations when controls use exactly the same token positions and matched perturbation norms? This would help distinguish region-specific functional importance from sensitivity to the intervention itself.

  3. What evidence distinguishes the proposed regions from clusters driven by sentence content, reasoning position, or surface form? Given the label-permutation results and limited behavior-specific controllability, which mechanistic conclusions do the authors consider directly supported?

Addressing these concerns with clarifications and supporting evidence could materially strengthen the paper, and I would be open to revising my recommendation upward.

Comments on the paper

1 comment

Dear Reviewers,

As the rebuttal period is drawing to a close, we kindly ask everyone to engage with our comments. We have addressed the questions and critiques and would appreciate any feedback going forward. Thanks for your time and attention,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.