acceptodds
Under review as a conference paper at ICLR 2027

Covering Language Model Outputs with Simplicial Complexes

Abstract

Language models (LMs) predict the next-token probability distribution over a large vocabulary, yet most of the probability mass is often concentrated on a small, context-dependent subset. This raises the question of whether LM outputs across contexts admit a compact representation. To answer this question, we first characterize this behavior of LM outputs using Bhattacharyya geometry on probability simplexes and simplicial complexes from algebraic topology. Our theorem connects probability mass discarded by truncation to our geometric dissimilarity measure. Because each output point lies near the low-dimensional simplex of its high-probability tokens, the output points are covered by a neighborhood of a simplicial complex, a union of simplexes. Next, we evaluate representations of LM outputs through the tradeoff among four metrics: . The first two measure the complexity of simplicial complexes, while the latter two assess approximation quality. To construct compact covers, we first consider collecting pointwise covers. For a single output, Top- and Top- truncation determine optimal simplexes in the tradeoff between and with fixed and . Collecting these simplexes yields simplicial complexes, which contain a prohibitively large number of simplexes in practice. To obtain a more compact complex, we formulate the construction task as a monotone submodular maximization problem and propose an algorithm for this problem. The algorithm provides the same worst-case guarantee as previous work. In the experimental section, we evaluate the construction using a natural language corpus (Dolma) and real-world LLMs (OLMo, Pythia). One of the resulting simplicial complexes uses 843 simplexes of dimension at most 4096 to cover 95% of the evaluated outputs while retaining at least 90% of each covered output's probability mass, providing a compact representation of approximately 50K-dimensional LM outputs over a 100M-token corpus.

Primary area:
interpretability and explainable AI
Keywords:
Language Models, Geometry, Softmax, Topology, Simplicial Complex, submodular function

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QXZXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper approximates language-model outputs with simplicial complexes. One cover uses 843 simplexes for 95% of tested outputs. Each covered output retains at least 90% probability mass.

Critical strengths

The squared geometric error equals discarded probability mass. This makes the error easy to interpret.

Critical weaknesses

Implementation errors prevent application of the proved guarantee to the tested code. The comparisons lack shared-cover baselines under equal size limits. The method’s added value remains unclear.

Questions for authors

Do corrected runs preserve coverage? Does the method outperform random shared covers under equal limits?

1 comment

Thank you for asking whether the implementation correction affects coverage and how our covers compare with random covers of comparable size. Your questions helped us show how much coverage changed after the correction and how our method compares with random shared covers under the same size limits.

1. Regarding results by corrected implementation

Do corrected runs preserve coverage?

Short response: We reran the experiments with corrected implementations and confirmed that the results remain largely unchanged.

Your concern about the mismatch between the guarantee and the implementation is justified. Repeating the experiments with the corrected procedure is necessary to determine whether the reported findings still hold.

We corrected the threshold initialization and repeated the evaluation over the same parameter grid for OLMo and Pythia. The minimum target threshold is now strictly positive, and the initial supports are always included in the candidate stream.

The results remain largely unchanged: based on the reported values, coverage changes by no more than 0.1 percentage points in 49 of 54 settings. In the remaining five settings, coverage improves by 1.2–2.2 points for OLMo and 3.3–3.7 points for Pythia. No setting shows a decline in coverage at this precision.

The corrected evaluation therefore preserves the empirical findings and addresses the concern that they depend on the flawed implementation. We will revise the figures and implementation description accordingly.

2. Regarding comparison with randomly constructed simplicial complexes

Does the method outperform random shared covers under equal limits?

Short response: Yes, the simplicial complexes constructed by our method outperforms randomly constructed supports.

We agree that comparisons with pointwise Top-k/Top-p constructions alone do not establish the benefit of selecting shared supports. This requires comparison with simple shared covers under comparable budgets.

Following your suggestion, we constructed baseline simplicial complexes from 1) randomly selected admissible Top-p supports, 2) supports sampled from a unigram-weighted distribution, and 3) supports sampled uniformly over the vocabulary. These constructions used the same nominal dimension and simplex-count limits. We then evaluated all covers on held-out Dolma documents that had not been used to construct any of these complexes.

Our covers outperform the random Top-p baseline with the same construction threshold in all 81 comparisons per model (27 settings and three evaluation thresholds). Table 1 compares the methods at the central grid setting for OLMo, where our cover achieves the highest coverage among the listed baselines.

These comparisons provide evidence that coverage-based selection using Sieve-Streaming improves on simple shared-cover constructions. The observed benefit therefore goes beyond the pointwise concentration captured by Top-k/Top-p.

Table 1. OLMo coverage at a representative parameter.

CoverSimplexesMax. dimensionTrainHeld-out
Ours2564,09692.13%91.68%
Random Top-p (.90)2563,66764.54%64.71%
Unigram-weighted2564,09667.12%66.76%
Uniform2564,09635.11%34.84%
Single Top-p (.90)13,86422.44%22.70%
Top unigram14,09655.59%55.54%
Single uniform14,0962.36%2.47%

*Single Top-p and single uniform: column-wise medians over 16 draws.

Your questions helped us notice which comparisons were missing, and the results of those comparisons now lend more direct support to our conclusions.

Reviewer_cHCROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether next-token distributions across contexts can be approximated by a compact collection of vocabulary subsets. It represents each subset as a probability simplex and their union as a simplicial complex. The main geometric result shows that the squared Bhattacharyya dissimilarity from a distribution to a vocabulary simplex equals the probability mass discarded by truncation to that subset. The paper characterizes covers through maximum simplex dimension, simplex count, approximation radius, and uncovered fraction, and adapts Sieve-Streaming to select a compact collection of supports.

Critical strengths

The connection between discarded probability mass and distance to a vocabulary simplex gives the geometric formulation a direct probabilistic interpretation. Tracking dimension, simplex count, approximation error, and uncovered fraction together distinguishes pointwise concentration from reuse across contexts.

The authors provide substantial empirical demonstration of shared supports and the results demonstrate that a relatively small collection of vocabulary subsets can approximate many observed outputs.

Critical weaknesses

The baselines do not establish how informative the reported compactness is. Collecting distinct Top-k or Top-p supports does not optimize sharing across contexts, so reducing the number of supports relative to these constructions is insufficient to establish the effectiveness of the proposed selection procedure. Comparisons with randomly sampled admissible supports, a global frequent-token subset, or a clustering-based construction at comparable budgets would help determine how much of the result reflects nontrivial cross-context structure rather than common-token concentration.

The approximation guarantee is disconnected from the reported implementation. The authors concede that the reported constructions may not satisfy the bound. Since the guarantee is part of the algorithmic contribution, the paper should evaluate the corrected algorithm or consistently qualify the experimental method as a heuristic.

Questions for authors

How does coverage compare with randomly sampled admissible Top-p supports and a global frequent-token subset under comparable dimension and simplex-count budgets?

Do covers constructed from one collection of documents retain similar coverage on disjoint documents? The current same corpus evaluation supports a result about that corpus, but leaves the broader generality of the shared structure unclear.

1 comment

Thank you for pointing out the mismatch between the algorithm and its implementation and for asking about additional experiments. Your comments helped us correct the implementation and design comparisons that test both the benefit of support selection and whether the selected supports generalize to new contexts.

1. Regarding comparisons with baselines

How does coverage compare with randomly sampled admissible Top-p supports and a global frequent-token subset under comparable dimension and simplex-count budgets?

Short response: Simplicial complexes constructed by our method outperform randomly constructed supports.

We agree that comparisons with pointwise Top-k/Top-p constructions alone do not establish the benefit of selecting shared supports. This requires comparison with simple shared covers under comparable budgets.

Following your suggestion, we constructed baseline simplicial complexes from 1) randomly selected admissible Top-p supports, 2) supports sampled from a unigram-weighted distribution, and 3) supports sampled uniformly over the vocabulary. These constructions used the same nominal dimension and simplex-count limits. We then evaluated all covers on held-out Dolma documents that had not been used to construct any of these complexes.

Our covers outperform the random Top-p baseline with the same construction threshold in all 81 comparisons per model (27 settings and three evaluation thresholds). Table 1 compares the methods at the central grid setting for OLMo, where our cover achieves the highest coverage among the listed baselines.

These comparisons provide evidence that coverage-based selection using Sieve-Streaming improves on simple shared-cover constructions. The observed benefit therefore goes beyond the pointwise concentration captured by Top-k/Top-p.

Table 1. OLMo coverage at a representative parameter.

CoverSimplexesMax. dimensionTrainHeld-out
Ours2564,09692.13%91.68%
Random Top-p (.90)2563,66764.54%64.71%
Unigram-weighted2564,09667.12%66.76%
Uniform2564,09635.11%34.84%
Single Top-p (.90)13,86422.44%22.70%
Top unigram14,09655.59%55.54%
Single uniform14,0962.36%2.47%

*Single Top-p and single uniform: column-wise medians over 16 draws.

2. Regarding results by corrected implementation

the paper should evaluate the corrected algorithm or consistently qualify the experimental method as a heuristic.

Short response: We reran the experiments with corrected implementations and confirmed that the results remain largely unchanged.

Your concern about the mismatch between the guarantee and the implementation is justified. Repeating the experiments with the corrected procedure is necessary to determine whether the reported findings still hold.

Following your suggestion, we constructed the threshold initialization and repeated the evaluation over the same parameter grid for OLMo and Pythia. The minimum target threshold is now strictly positive, and the initial supports are always included in the candidate stream.

The results remain largely unchanged: based on the reported values, coverage changes by no more than 0.1 percentage points in 49 of 54 settings. In the remaining five settings, coverage improves by 1.2–2.2 points for OLMo and 3.3–3.7 points for Pythia. No setting shows a decline in coverage at this precision.

The corrected evaluation therefore preserves the empirical findings and addresses the concern that they depend on the flawed implementation. We will revise the figures and implementation description accordingly.

3. Regarding evaluation by held-out data

Do covers constructed from one collection of documents retain similar coverage on disjoint documents?

Short response: Yes, the coverage remained similar on the unseen contexts

As you point out, evaluation on the construction data alone cannot establish whether the selected supports are reusable on unseen data. An independent evaluation is needed.

We kept the OLMo covers fixed and expanded the evaluation to five files each of construction-side and held-out data, with 10M contexts in each group. None of the held-out documents had been used to construct any experimental cover.

Across the 81 comparisons of covers and evaluation thresholds, coverage remained similar on the unseen contexts, with a maximum absolute difference of 0.93 percentage points. As shown in Table 1, the cover at the representative setting achieves 92.13% coverage on construction-side data and 91.68% on held-out data.

These results address the concern that coverage is specific to the construction data: the same OLMo supports retain similar coverage on the separate held-out portion of the corpus.

Your comments prompted us to broaden our evaluation, and the additional results provide stronger support for the claims we make in the paper.

Reviewer_cXZ5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new analysis framework for the decoding token probability distribution of LLMs. Specifically, they introduce a tool from Bhattacharyya geometry, which aims to model the token prediction probability space as simplexes and simplicial complexes. They then measure the output space with the four metrics under this scope and propose both theoretical and empirical investigations.

Critical strengths
  • I think this is really an interesting view for the LLM output distribution analysis, though I think it is not enough.
  • This work is generally solid with many theoretical analyses.
  • Several empirical experiments are provided.
Critical weaknesses
  • My core concern: currently, the analysis is at the single-token level. This only corresponds to the traditional single-token classification paradigm. The complexity of LLMs, coming from auto-regressive generation, constructs a multi-token combination of the output space, which in my opinion is why the output sentences are complex and can support so many different and complex tasks, and is the reason for non-familiar token/phrase analysis. Without considering it, I personally think it cannot be regarded as an LLM analysis work, and in practice is also just a toy.
  • The experiments are not enough. The settings are very thin. The experiments are also not enough, especially without any implications.
  • Lack of any application cases.
Questions for authors

Please refer to the weaknesses.

1 comment

Thank you for asking how our token-level analysis relates to generation and what our experiments show.

1. Regarding token-level analysis

My core concern: currently, the analysis is at the single-token level.

Short response: While autoregressive structure plays an important role in analysing language-model behavior, token-level analysis remains an important tool for understanding and improving language models.

We share your view that processing token sequences and capturing their dependencies are central to language models' ability. Yet, at the same time, token-level analysis helps explain how information from a sequence is represented and used in each prediction.

Prior work uses this approach to study language-model representations. For example, symmetries in word co-occurrence statistics help explain their geometry. Karkada et al., 2026. Anthropic's work on J-space identifies verbalizable internal representations that support reasoning without being emitted as text. The authors also discuss applications to alignment auditing and improving model behavior. Gurnee et al., 2026. These studies support the value of analysing single-token.

While multi-token or autoregressive behavior is important characteristics of language models, single-token analysis remains an essential area of research, as the single-token prediction is the fundamental unit of the generation process.

2. Regarding experimental scope and implication

The experiments are not enough.

Short response: We have conducted two additional experiments: 1) comparisons with randomly constructed baselines and 2) generalization evaluation on held-out contexts.

Your comments prompted us to ran two additional experiments. These experiments also helped clarify the conclusion of our paper.

1). Comparison with randomly constructed baselines. We compared our covers with random admissible Top-p supports, unigram-weighted supports, and uniformly sampled vocabulary supports under the same nominal dimension and simplex-count limits. All 81 comparisons per model outperform the random Top-p baseline with the same construction threshold. Table 1 shows the comparison at the representative OLMo setting.

2). Generalization to held-out contexts. We kept the OLMo covers fixed and evaluated them on held-out contexts. Across all 81 comparisons, the maximum absolute coverage difference is 0.93 percentage points; see Table 1 for a representative comparison..

The shared structure constructed by our method captures conditional next-token distributions more effectively than randomly constructed covers, and its coverage generalizes to held-out documents. We will include these additional results and clarify this implication in the revised paper.

Table 1. OLMo coverage at a representative parameter.

CoverSimplexesMax. dimensionTrainHeld-out
Ours2564,09692.13%91.68%
Random Top-p (.90)2563,66764.54%64.71%
Unigram-weighted2564,09667.12%66.76%
Uniform2564,09635.11%34.84%
Single Top-p (.90)13,86422.44%22.70%
Top unigram14,09655.59%55.54%
Single uniform14,0962.36%2.47%

*Single Top-p and single uniform: column-wise medians over 16 draws.

3. Regarding lack of Applications

Lack of any application cases.

Short response: The absence of a downstream application does not invalidate our main contribution, and the framework offers a basis for investigating the mechanisms underlying language-model behavior.

We agree that the current paper does not include a downstream application such as generation acceleration or task-specific performance improvement. The primary goal of this work is to introduce and validate a geometric characterization, rather than to propose an application.

We would like to clarify that a downstream application is not required to establish the validity of the claims made in this paper. Our theoretical claims concern the relation between probability truncation and Bhattacharyya geometry, while our empirical claims concern cross-context compactness. A downstream application would be valuable, but it would test an additional claim about the utility of the framework for a particular task rather than the soundness of these characterization results.

We also agree that the potential uses and practical implications of the framework should be made clearer. Two natural uses are: (i) using the (d,m,r,u)(d,m,r,u) frontier as a diagnostic for comparing how concentrated and reusable the output supports of different models or domains are, and (ii) using the selected vocabulary simplexes as reusable analysis units for studying context-dependent output structure, domain shifts, or autoregressive trajectories. Our current OLMo/Pythia experiments provide an initial illustration of the first use.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.