acceptodds
Withdrawn from ICLR 2027

Typicality of Representations Predicts Input-Level Performance in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting whether a model will perform well on a set of inputs remains a topic of central importance in machine learning. Motivated by the information-theoretic notion of typicality, we address this open problem by introducing representational typicality: a framework for measuring how well the intermediate representations induced by an input conform to a model's expected representational regularities, and consequently of the model's performance on the input. Without requiring any dataset labels or model retraining, the proposed method first derives the characteristic statistics of a model's intermediate representations using only readily accessible data. The method then evaluates whether the representations induced by new data are compatible, in the sense of being typical, with these characteristic representation statistics to predict input-level performance. Across an array of widely used language models and downstream tasks, our experiments comprehensively demonstrate that typicality is highly predictive of input-level performance, outperforms baseline measures, remains predictive across varying experimental configurations, and is robust to a suite of hypothetical confounding factors.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Typicality, Intermediate Representations, Large Language Models, Input-Level Performance Prediction, Information Theory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9bUEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether a language model's hidden representations can help predict if it will answer a given input correctly. The authors propose a simple metric called typicality distance, which measures how different an input's hidden representations are from those of a reference dataset. The method does not require labeled data or additional model training.

Experiments on different language models and benchmarks show that inputs with smaller typicality distances are more likely to be answered correctly. The authors also compare their method with existing baselines and study how different factors, such as dataset size, model layers, input length, and statistical assumptions, affect the results.

Critical strengths
  1. The paper applies information-theoretic typicality to hidden representations, providing a different way to study when language models are likely to produce correct answers.

  2. The proposed approach requires no labeled calibration data, additional training, or auxiliary models. The consistent trends across different models and tasks suggest that representational typicality is a useful direction for further study.

Critical weaknesses
  1. The paper divides examples into five buckets and measures the correlation between typicality distance and bucket-level accuracy. Even a perfect correlation across five buckets does not necessarily indicate reliable predictions for individual inputs. Since input-level prediction is the paper's central claim, metrics such as AUROC, AUPRC, and risk-coverage curves would provide stronger evidence.

  2. Most experiments use calibration and test data from the same benchmark. It is unclear whether the method works across different datasets or distribution shifts.

  3. The paper uses different activation normalization methods during calibration and testing, without analyzing their effects. More discussion of numerical stability, computational cost, and comparisons with stronger uncertainty estimation baselines would improve the evaluation.

Questions for authors
  1. Can the authors report AUROC, AUPRC, and risk-coverage curves to show how well typicality predicts correctness for individual inputs, rather than grouped examples?

  2. Does the method still work when calibration and test data come from different datasets? How does distribution shift affect the results?

  3. Why are different standardization methods used during calibration and testing? How sensitive are the results to this choice and covariance regularization?

Reviewer_cGYbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces representational typicality to predict input-level performance in language models. It measures how well intermediate representations match the statistics of a calibration set. The proposed typicality distance requires no dataset labels or model retraining. Experiments across five language models and seven downstream tasks show strong correlations with performance.

Critical strengths
  1. The information-theoretic notion of typicality provides an interesting perspective on input-level performance.

  2. The method requires no dataset labels or model retraining. The paper includes baseline comparisons, ablations, and control experiments.

Critical weaknesses
  1. Limited input-level evaluation. Performance is measured across five buckets rather than individual inputs. The reported correlations may depend on this evaluation protocol.

  2. Limited novelty. The proposed typicality distance builds on existing information-theoretic typicality and Mahalanobis distance. Its additional contribution over existing representation-space statistics remains unclear.

  3. Limited generalization. Calibration and evaluation data come from the same dataset. Generalization across different distributions remains unclear.

Questions for authors
  1. How well does typicality distance predict correctness for individual inputs, without dividing the validation set into five buckets?

  2. What is the advantage of typicality distance over simpler representation-space statistics, such as Mahalanobis distance?

  3. How robust is the proposed method when the calibration set and test inputs come from different datasets?

Reviewer_Jz37OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a notion of representational typicality as a way to predict language model performance on individual inputs or a set of new inputs. The method estimates hidden layer representation statistics from an unlabeled calibration set, then measures how much a new input’s representations deviate from an expected ‘’typicality’’ level. They aggregate these deviations across tokens and layers to obtain an input-level score. Experiments show that validation inputs grouped by lower typicality distance generally have higher accuracy.

The definitions are clear and there is some evidence, but I would consider the empirical study here to be more of a workshop paper level contribution rather than a full paper, since there appears to be some holes in the experiments.

Critical strengths

The introduction provides an understandable motivation and presentation for studying typicality, and the methods section clearly describes the calibration procedure and scoring metric.

Estimating reliability from internal activations without labeled calibration data or retraining could be valuable.

The reported plots show substantial accuracy differences between groups of more and less typical inputs, motivating further investigation into this metric.

Critical weaknesses

Different experiments use different combinations of models and benchmarks, making it difficult to assess consistency across settings. For example, the baseline comparisons in Section 4.3 largely use different pairings from those in Figure 1. Each model also uses different benchmarks. Please explain how these combinations were selected, whether other combinations were evaluated, and whether any results were omitted. Without this information, I am concerned that the displayed settings may disproportionately represent cases where the method works well. Evaluating a common set of benchmarks across all models and reporting the full evaluated grid, would strengthen the evidence.

Section 4.3 compares against several alternatives, but additional representation-based metrics would help establish the value of the proposed score. In particular, it would be useful to consider relevant metrics such as prompt entropy and effective rank studied in (Skean et. al, 2025) , or explain why they are unsuitable for this setting. Although they are not training free, it would be good to also compare against LLM-Check and Semantic Entropy Probes.

The evaluation divides inputs into only five equally sized bins and reports Spearman correlations across the resulting five points. A high Spearman correlation over five aggregated points provides limited information about predictive performance at the individual-input level. The authors should show results with more bins and examine sensitivity to the bin count, including the uncertainty associated with smaller groups. This would make the strength and consistency of the relationship easier to assess.

Comparing the last third, last two thirds, and all layers does not reveal how predictive typicality is at individual depths. Computing the score separately for each layer and plotting predictive performance against layer depth would show where the signal is strongest and whether aggregation hides substantial variation.

Questions for authors
  1. Why do the model–benchmark pairings change between Figure 1 and Section 4.3, and why are different models evaluated on different benchmarks? Can you report results on a fixed set of benchmarks across models, and provide the full evaluated grid, to clarify whether the displayed results are representative?

  2. Does the relationship between typicality and accuracy remain consistent with more than five bins?

  3. How does the predictive performance of typicality vary across individual layers?

  4. Is the typicality measure something that can improve accuracy over a benchmark, let's say if I am using it as a way to route a prompt to multiple LLMs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.