acceptodds
Under review as a conference paper at ICLR 2027

The Low-Rank Canonical Geometry of LLMs

Abstract

Large language models predict the next token from vocabularies exceeding 10^5 tokens, suggesting an extremely high-dimensional output space. Yet it remains unclear whether their predictive distributions exploit this enormous space, or instead vary within a much smaller structure as context varies. We develop a spectral framework that characterizes this geometry directly in probability space. By separating the model's predictive marginal from context-dependent probability variation, the framework decomposes LLM next-token prediction into orthogonal token-side canonical modes, revealing both the dimensionality and orientation of the predictive geometry. Surprisingly, we find that LLM prediction is organized around a much smaller canonical geometry despite their vast vocabulary. Across model families and context distributions, nearly all context-dependent predictive variation is captured by a small set of dominant token modes: only 9.7%–24.0% of the token-space dimensions account for \(99.9%\) of this variation. More surprisingly, this geometry remains largely intact under downstream adaptation. Fine-tuning changes the model's conditional predictions while largely preserving the concentration of the canonical spectrum and the dominant canonical token subspace, even across different downstream tasks. These observations motivate the Canonical Subspace Hypothesis: pretrained language models organize next-token prediction around a low-dimensional canonical token geometry that downstream adaptation largely reuses rather than reconstructs. More broadly, our framework provides a general tool for studying how the geometry of LLM prediction is formed, preserved, and transformed throughout training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Next-Token Prediction, Low-Rank Structure, Predictive Geometry, Canonical Subspace, Spectral Analysis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QrbDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+2)
Read review
Paper summary

This paper presents a method for understanding LLM output token probability distributions from a geometric perspective. The authors decompose a model's next-token probability vector and remove the context-independent component to isolate the context-specific variation in token probability over a dataset of 1M tokens. They then use an eigendecomposition of this context-specific distribution to find that 99.9% of variation in that distribution can be explained by ~10-24% of the available dimensions. They further show that this low-rank distribution does not substantially change during model fine-tuning.

Critical strengths
  • The paper is very well-written and easy to follow. The theoretical proofs cleanly support the empirical experiments and the analysis was similarly well organized.
  • The mathematical theory behind their spectral framework is well-founded and thorough. The proofs in the main text provide enough explanation to follow their methods, and the appendix provides substantial support for those interested.
  • The authors were careful in outlining the separation of their methods from logit rank, and in demonstrating that their results cannot be directly attributed to architectural components.
Critical weaknesses
  • The authors never acknowledge a simple explanation for their results, which is the output token frequency distribution. Although I recognize that the methods in the paper are looking at the entire output probability distribution (not limited to the argmax tokens), the simplest explanation for their findings is that it is a direct representation of the frequency distribution in their dataset.
  • Given the near-Zipfian distribution of language, a sample of 1M tokens is too small to reliably capture the probability distribution over a 100K+ token vocabulary. In this size dataset, we would only expect to see a few thousand tokens more than a handful of times, and a huge number of rare tokens would never appear at all. Even with a large enough sample, we would expect to capture 99.9% of the token frequency distribution in just 35-60% of a 100-200K token vocabulary. A small dataset that cuts off the long tail would skew that proportion, making the ~10% concentration shown in this paper a possible side effect of the dataset alone.
  • The fine-tuning results could similarly be explained by term frequency--if the fine tuning dataset frequency distribution is similar to that of the pre-training dataset, we would not expect to see a difference in the output distribution after fine tuning. The separation of those results on the ELI5 task actually seems to support this further, since ELI5 likely has a more limited vocabulary than GSM8K and would, thus, cause a model to produce a lower number of tokens (and a "tighter" probability distribution).
  • I'm not sure that the random-model ablation demonstrates that the low-dimensional geometry is learned. It is difficult to see the random line on the plots, but I am reading this as evidence that context-dependence is learned. If the low-dimensional geometry were learned, wouldn't we expect to see the model using more of the probability distribution before training? Instead, we see that a few directions capture all of the context-dependent information before training, and that signal is spread out across more directions after training.
Questions for authors

Clarifications that could change my vote:

  • I want to see some evidence or clear explanation that would distinguish your results from word frequency statistics. Even if you find that your spectral framework is capturing the same information as word frequency, that needs to be acknowledged and addressed.
  • Similarly, I would like to see an explicit acknowledgment of the issues that your dataset size introduces. I recognize that compute is not always accessible, so the small dataset size is not an issue as long as you include some discussion of how that changes your ability to draw conclusions.

Very minor suggestions for a revision:

  • It seems like you are using "canonical" in the sense of a canonical correlation analysis (CCA). It would be helpful if you state this (or otherwise provide a definition) early on, as there are several meanings for "canonical" and you use it throughout the paper.
  • The dashed line in Figure 2 and Appendix B is basically not visible--think about how to fix these so that they are clearer.
4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

We thank the reviewer for raising the possibility that the observed spectral concentration could be explained by token-frequency statistics. We believe this interpretation mixes two distinct issues: finite-sample token counts and the model's predictive marginal qθq_\theta. Neither accounts for the canonical spectral behavior we observe.

The canonical geometry is defined jointly by the model pθp_\theta and probing distribution μ\mu, so we do not claim a universal, dataset-independent rank. The question is instead whether the concentration of the canonical spectrum is simply inherited from the concentration of qθq_\theta. Our additional analysis in Appendix E, Figure 11 shows that it is not.

1. The 10610^6-context estimate is not a 10610^6-token frequency histogram.

A central part of the reviewer's concern appears to rely on the analogy with a corpus containing 10610^6 sampled tokens, where rare vocabulary items may occur only a few times or not at all. This is not how our estimator is constructed. As described in Section 2.2, every one of our 10610^6 contexts contributes the complete next-token probability vector

pθ(⋅∣hi)∈RV.p_\theta(\cdot\mid h_i)\in\mathbb{R}^{V}.

The predictive marginal is therefore

= \frac{1}{N} \sum_{i=1}^{N}p_\theta(w\mid h_i),$$ which averages model probabilities rather than observed next-token counts. Consequently, every vocabulary coordinate contributes $$p_\theta(w\mid h_1),\ldots,p_\theta(w\mid h_{10^6})$$ to the analysis, including tokens that are never generated. For Qwen, **each of the $152064$ vocabulary coordinates receives approximately $10^6$ probability evaluations**. The observed spectral concentration is therefore not caused by rare vocabulary coordinates being absent from a $10^6$-sample token histogram. **2. The concentration of $q_\theta$ does not determine the concentration of the canonical spectrum.** We directly test the proposed explanation by plotting the ranked predictive marginal $q_\theta(w)$ over the complete vocabulary in **Appendix E, Figure 11**. Its behavior tells a qualitatively different story from the canonical spectrum. Two comparisons make this especially clear: - **Random initialization $\rightarrow$ pretraining:** the random model has a much broader predictive marginal---approximately $151K}$ of $152064$ tokens are required to capture $99.9\%$ of its marginal mass. Pretraining makes $q_\theta$ dramatically more concentrated. Yet the canonical spectrum changes in the opposite direction: the random model has a **nearly degenerate spectrum**, by which we mean that almost all eigenvalues are negligible and essentially all spectral energy is confined to a very small number of leading modes. After pretraining, a much broader set of nontrivial canonical modes becomes active (**Figure 2 and Appendix B, Figure 6**). - **Pretraining $\rightarrow$ ELI5 fine-tuning:** on ELI5 contexts, fine-tuning again makes $q_\theta$ more concentrated. In this case, however, the canonical spectrum also becomes slightly more concentrated, exhibiting an earlier decay. Thus, **the same change in the marginal---increased concentration of $q_\theta$---is associated with opposite changes in spectral concentration in the two settings**. From random initialization to pretraining, a more concentrated marginal accompanies a *broader* canonical spectrum; from pretraining to ELI5 fine-tuning, a more concentrated marginal accompanies a *more concentrated* spectrum. Therefore, the concentration of $q_\theta$ alone cannot explain the canonical spectral behavior. This is also clear from the definitions. The marginal $$q_\theta(w)=\mathbb{E}_{h\sim\mu}[p_\theta(w\mid h)]$$ describes where probability mass lies on average, whereas $$C_\theta = \mathbb{E}_{h\sim\mu} \left[ z_\theta(h)z_\theta(h)^\top \right]$$ describes how that probability distribution changes as context varies. For an arbitrary, highly non-uniform $q_\theta$, if $p_\theta(\cdot\mid h)=q_\theta$ for every context, then $C_\theta=0$. Hence $q_\theta$ is part of the canonical construction, but does not determine its eigenspectrum. This also clarifies the random-initialization result. The nearly degenerate spectrum of the random model should not be interpreted as a learned low-dimensional representation. It reflects the near absence of structured context-dependent prediction. Pretraining creates substantial context-dependent predictive variation across many modes; the nontrivial observation is that this learned variation nevertheless remains confined to a restricted fraction of the available token space. We have revised the discussion around **Figure 2** to make this interpretation explicit.
AuthorsOpenReview ↗

Official Comment by Authors (2/2)

3. The ELI5 result is not explained by a smaller effective predictive vocabulary.

The reviewer additionally suggests that the ELI5 behavior may arise because ELI5 uses a smaller vocabulary. Here it is important to distinguish the realized tokens in an ELI5 corpus from the model's predictive distribution. At every ELI5 context, our estimator evaluates all 152064 next-token probabilities; the analysis is therefore not restricted to the set of tokens actually appearing in ELI5 outputs.

Moreover, the predictive marginal itself is not uniformly narrower on ELI5. For the pretrained model, 99.9% of marginal probability mass requires 32503 tokens on ELI5, compared with only 11544 on GSM8K (Appendix E, Figure 11). Thus, ELI5 does not simply correspond to probing the model over a smaller predictive vocabulary.

Finally, we thank the reviewer for the presentation suggestions. We have clarified the meaning of “canonical,” made the dependence on the probing distribution explicit, clarified the interpretation of random initialization, and added the predictive-marginal analysis in Appendix E, Figure 11.

AuthorsOpenReview ↗

Rebuttal reminder

Hi,
Just a gentle reminder as the discussion deadline is approaching. We hope our response and the additional clarifications/experiments addressed your concerns, and we would be grateful if you could reconsider your assessment in light of our revised response. We would also greatly appreciate any further feedback you may have before the deadline. Thank you again for your time and consideration.

Thank you to the authors for your response to my concerns.

Appendix E Thank you for the additional analysis here. This work addresses most of my concerns about the relationship between token frequency (or your predictive marginal distribution) and your canonical spectrum. In particular, your demonstration of a disconnect between qθq_{\theta} and CθC_{\theta} on the fine-tuning and random initialization experiments provide sufficient evidence that the two distributions are not causally linked. I also appreciate the acknowledgment that they are still related.

A few minor concerns remain:

As described in Section 2.2, every one of our (10^6) contexts contributes the complete next-token probability vector

I understand that this is the case, but we would expect little variation in the probability estimates for rare tokens, especially in a relatively limited dataset. I still think that it is worth acknowledging that some of your low-rank findings may be directly related to the frequency distribution of language more generally.

We have revised the discussion around Figure 2 to make this interpretation explicit.

I am not seeing a change in your revised manuscript here. I think this does deserve one or two additional sentences to make the distinction clear.

In light of the authors' response and revisions, I would like to update my recommendation to Weak Acceptance.

Reviewer_AspPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The research team studies the canonical geometry of LLM next-token prediction and is trying to understand if their context-dependent variation is concentrated in a lower-dimensional subspace despite the large vocabulary size. To study these properties, the team developed a spectral framework for characterizing LLM prediction directly in probability space. The authors then find a low-dimensional predictive geometry across model families and context distributions using their proposed pipeline. The authors evaluate the proposed framework on Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct using GSM8K and MMLU, finding that only 9.7%–24.0% of vocabulary dimensions account for 99.9% of the context-dependent predictive variation. By comparing pretrained and randomly initialized models, they suggest that this structure emerges during training. They further investigate downstream adaptation by fine-tuning Qwen2.5-14B-Instruct on GSM8K and ELI5, observing that both the spectral concentration and dominant token subspace remain largely preserved. These findings motivate their Canonical Subspace Hypothesis, suggesting that fine-tuning mainly reuses the predictive geometry learned during pretraining.

Critical strengths
  1. The research team proposed a well-defined pipeline to study next-token prediction in probability space. This is very interesting, as the researchers are not trying to study the hidden state or logit rank, which is commonly studied in other works. The authors constructed the covariance operator based on the marginal-normalized probability deviations, and use its eigenvalues and eigenvectors to describe a canonical spectrum Λθ\Lambda_\theta and a canonical token subspace UθU_\theta。
  2. The paper separates spectral concentration from subspace preservation. This is mathematically reasonable.
Critical weaknesses
  1. The fine-tuning preservation experiment is not strong enough to support the Canonical Subspace Hypothesis. In Section 3.4, the authors only fine-tune Qwen2.5-14B-Instruct for 100 optimization steps using LoRA, without reporting the corresponding downstream performance changes. It is therefore unclear whether the model has undergone substantial adaptation. If the model's behavior has not changed much, the high subspace overlap in Figure 4 may not be particularly surprising.

  2. The canonical geometry depends on both the model parameters θ\theta and the context distribution μ\mu. The authors estimate the geometry using autoregressive contexts generated separately by the pretrained and fine-tuned models. Therefore, they are effectively comparing Cθbase,μbaseC_{\theta_{\mathrm{base}},\mu_{\mathrm{base}}} with Cθft,μftC_{\theta_{\mathrm{ft}},\mu_{\mathrm{ft}}}, where both θ\theta and μ\mu may have changed. I think it would be useful to also compare Cθbase,μC_{\theta_{\mathrm{base}},\mu} and Cθft,μC_{\theta_{\mathrm{ft}},\mu} under the same fixed context distribution. This would help clarify whether the observed subspace preservation is due to the model retaining its predictive geometry, rather than being affected by changes in the generated contexts.

Questions for authors

same as above

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for highlighting two important questions: whether canonical preservation persists beyond the step-100 checkpoint, and whether it remains when the probing context distribution is held fixed. We added new experiments addressing both points.

Q1. What do the existing fine-tuning or validation records show about how much the model changed?

The step-100 checkpoints already undergo substantial adaptation. On GSM8K, validation loss decreases from (2.3030) to (0.8157), perplexity from (10.00) to (2.26), and token accuracy increases from (68.24%) to (82.15%). On ELI5, validation loss decreases from (3.9474) to (1.7361), perplexity from (51.80) to (5.68), and token accuracy increases from (42.89%) to (61.51%). These results and the corresponding optimization curves are reported in Appendix D.1, Figure 7 and Table 3--4.

More importantly, preservation is not limited to this single training point. We extend GSM8K fine-tuning to 800 optimization steps and re-estimate the canonical geometry at steps 50, 100, 200, 400, and 800.

Despite substantial changes in optimization and task performance (Figure 8; Table 5), the canonical spectrum remains strongly concentrated (Figure 9) and the dominant canonical subspace remains highly aligned with the pretrained model throughout training (Figure 5, left). For example, task-level GSM8K accuracy changes from (84.31%) for the pretrained model to (89.61%) at step 50 and (85.90%) at step 800.

Thus, canonical preservation persists across checkpoints with meaningfully different predictive and task-level behavior. This illustrates a central benefit of the proposed framework: it allows changes in model behavior to be distinguished from changes in the geometry carrying those predictions. The extended trajectory provides further support for the Canonical Subspace Hypothesis and for the interpretation that fine-tuning primarily adapts prediction within a geometry inherited from the foundation model.

Q2. Can the preservation comparison be made on identical contexts?

Our original analysis intentionally evaluates each checkpoint under its own autoregressive context distribution. During actual generation, the contexts encountered by an autoregressive LLM are induced by its previous predictions. Evaluating each model along its own generated trajectories therefore measures the canonical geometry along the states that the model itself actually visits. This is the setting used in the primary fine-tuning analysis in Section 3.4 and Figure 4.

At the same time, this on-policy construction allows both the model and the induced context distribution to change. We therefore add a complementary fixed-context control. For both GSM8K and ELI5, we generate probing contexts once using the pretrained model and then evaluate the pretrained and corresponding task-fine-tuned models on exactly the same saved contexts.

Under this shared empirical context distribution, the preservation result remains strong and in fact becomes stronger. Figure 5 (middle–right) compares the original on-policy overlap with the fixed-context overlap on ELI5 and GSM8K. In both cases, the dominant fine-tuned subspace remains highly aligned with the foundation-model subspace, with higher overlap under fixed contexts. This is consistent with part of the on-policy reduction in overlap arising from changes in the autoregressive trajectories visited after fine-tuning. The fixed-context construction is described in detail in Appendix D.3.

The two evaluations are therefore complementary: the original on-policy analysis asks whether geometry is preserved along the trajectories each model actually generates, while the fixed-context analysis asks whether the same conclusion holds when the probing distribution is held exactly constant. Both support the same conclusion: downstream adaptation largely reuses, rather than reconstructs, the dominant canonical predictive geometry.

Thanks to the authors for the detailed response and additional experiments. My main concerns were whether the original 100-step fine-tuning led to meaningful changes in model behavior, and whether the observed subspace preservation would still hold under a fixed context distribution.

I think the new results address both concerns reasonably well. The reported improvements in validation performance suggest that the model has undergone substantial adaptation, and the extended fine-tuning experiments provide additional evidence that the canonical geometry remains stable across training checkpoints. I also appreciate the fixed-context experiment, which directly addresses my concern about the changing context distributions and shows that the subspace overlap remains high even when the same probing contexts are used.

While the analysis is still limited to a relatively small set of models and adaptation settings, I find the additional evidence convincing and have a more positive view of the paper after the rebuttal.

Reviewer_CUPROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether context-dependent changes in LLM next-token probabilities occupy a much smaller space than the full vocabulary. It subtracts the predictive marginal, normalizes each token coordinate by the square root of its marginal probability, and eigendecomposes the covariance of these normalized deviations. The eigenvalues measure how concentrated the variation is, while the eigenvectors describe the corresponding token-space directions. Experiments across model families and context distributions report that 9.7%–24.0% of the token-space dimensions capture 99.9% of this variation. A LoRA fine-tuning study then compares the spectra and dominant subspaces before and after adaptation. The reported overlap motivates the Canonical Subspace Hypothesis: downstream adaptation largely reuses predictive directions established during pretraining.

Critical strengths

The probability-space formulation is interesting and is clearly distinguished from a low-rank logit argument. The experiments include random-initialization controls and multiple energy thresholds. Comparing subspaces directly is also more informative than comparing eigenvalue curves alone.

Critical weaknesses

The preservation experiment uses one model, 100 steps of rank-64 LoRA, and a learning rate of 5e-6, but does not report how task performance or predictions changed. If adaptation barely changes the model, high subspace overlap would not be very surprising. Please explain what changed in the existing fine-tuning run, using any recorded validation results. Without evidence of a meaningful change, the general preservation claim is stronger than the experiment supports.

The comparison also changes more than the model weights. Each checkpoint generates its own contexts and uses its own marginal normalization. The paper states this explicitly, but it still makes the result hard to interpret. Evaluating both checkpoints on the same contexts would give a cleaner test of what fine-tuning preserves.

The main limitation is that the current comparison does not isolate what fine-tuning preserves. A fixed-context control would help; without it, the conclusion should be limited to the overlap observed under the reported setup.

Questions for authors
  1. What do the existing fine-tuning or validation records show about how much the model changed?
  2. Can the preservation comparison be made on identical contexts with a common normalization?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the careful review. We added two analyses addressing the main concerns: (i) fine-tuning dynamics up to 800 steps, together with optimization and task-performance metrics, and (ii) a fixed-context control that compares foundation and fine-tuned models on identical probing contexts.

Q1. What do the existing fine-tuning or validation records show about how much the model changed?

The step-100 checkpoints already undergo substantial adaptation. On GSM8K, validation loss decreases from (2.3030) to (0.8157), perplexity from (10.00) to (2.26), and token accuracy increases from (68.24%) to (82.15%). On ELI5, validation loss decreases from (3.9474) to (1.7361), perplexity from (51.80) to (5.68), and token accuracy increases from (42.89%) to (61.51%). We now report these results and the corresponding optimization curves in Appendix D.1, Figure 7 and Table 3--4.

We further extend GSM8K fine-tuning to 800 steps and analyze checkpoints at steps 50, 100, 200, 400, and 800. Despite substantial changes in optimization and task performance (Figure 8; Table 5), the canonical spectrum remains strongly concentrated (Figure 9) and the dominant canonical subspace remains highly aligned with the foundation model throughout training (Figure 5, left). For example, task-level GSM8K accuracy changes from (84.31%) for the foundation model to (89.61%) at step 50 and (85.90%) at step 800.

We view this as an important consequence of the proposed framework: substantial changes in predictive behavior can be separated from changes in the geometry carrying those predictions. Across both the original step-100 checkpoint and the substantially longer trajectory, fine-tuning changes model behavior while continuing to reuse the dominant canonical geometry established by the foundation model. This provides further support for the Canonical Subspace Hypothesis and strengthens the interpretation that downstream adaptation operates largely within, rather than reconstructs, the predictive basis of the foundation model.

Q2. Can the preservation comparison be made on identical commun contexts?

Our original analysis is intentionally on-policy: each model is evaluated on the autoregressive contexts induced by its own previous predictions, e.g., like in standard CoT generation. It therefore measures whether the canonical geometry is preserved along the trajectories each model actually visits (Section 3.4, Figure 4).

To isolate changes in the model from changes in the visited context distribution, we now include a complementary fixed-context control. For both GSM8K and ELI5, we generate probing contexts once using the pretrained model and evaluate the pretrained and corresponding task-fine-tuned models on exactly the same contexts, thereby fixing the empirical probing distribution (\mu_{\mathrm{fixed}}).

The result is even stronger: Figure 5 (middle--right) shows higher canonical-subspace overlap under fixed contexts than in the original on-policy setting. This is consistent with part of the reduction in on-policy overlap arising from fine-tuning changing the trajectories visited by the model, rather than from reconstruction of the underlying canonical geometry. Full details are provided in Appendix D.3.

Each model still uses its own predictive marginal qθ(x)=EH∼μfixed[pθ(x∣H)],q_\theta(x)=\mathbb{E}_{H\sim\mu_{\mathrm{fixed}}}[p_\theta(x\mid H)],

because (q_\theta) is part of that model's canonical decomposition; imposing a shared marginal would define a different operator. Thus, canonical preservation holds both along each model's natural autoregressive trajectories and when the probing contexts are held exactly fixed. However, they both lead to the same conclusion: downstream fine-tuning changes predictive behavior while largely reusing, rather than reconstructing, the dominant canonical predictive geometry.

AuthorsOpenReview ↗

Rebuttal reminder

Hi,
Just a gentle reminder as the discussion deadline is approaching. We hope our response and the additional experiments addressed your concerns, and we would be grateful if you could reconsider your assessment in light of our revised response. We would also greatly appreciate any further feedback you may have before the deadline. Thank you again for your time and consideration.

Comments on the paper

1 comment

Dear Reviewers,

Kind reminder that the deadline of the discussion period is Nov. 18 so please please read the reviewers' response and take a minute to write an acknowledgment and confirm your final evaluation, as soon as possible.

Thanks a lot for your help.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.