acceptodds
Withdrawn from ICLR 2027

Bayesian Forecasting of LLM Outputs from Pre-Generation Hidden States

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The stochasticity of next-token prediction in large language models (LLMs) implies that LLM outputs should be represented probabilistically. Yet for problem settings in which outputs take on a structured format, obtaining such probabilities can be costly, as this requires repeated generation in response to a user's query. In this paper we propose a computationally cheap alternative to approximate LLM-derived probabilities. Using pre-generation hidden state representations of an LLM, our method forms predictive distributions that govern predefined concepts of the output space, aiming to forecast empirically-derived LLM probabilities. The approximating nature of the approach, a consequence of limited information and lack of foresight in hidden states, implies that a proper treatment of uncertainty is essential if we wish to place confidence in the resulting distributions. To this end, we utilize Bayesian classifiers in order to decompose a predictive distribution into aleatoric uncertainty, namely the LLM stochasticity that we intend to model, and epistemic uncertainty, or the model uncertainty arising from a lack of evidence in the LLM hidden states. We ground our work in the domain of visualization specification generation, representative of a task for which outputs adhere to a predefined grammar, and output concepts, such as graphical marks and visual channels, are well-defined. Experimental results illustrate that hidden states encode substantial information required for forecasting visualization design concepts, where forecasting difficulty increases towards composite tasks that require both context understanding and response planning. Hidden states also provide meaningful evidence for estimating uncertainty, correlated with inherent difficulty of the given query, suggesting that uncertainty can act as a meaningful criteria for evaluating forecasting quality.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
large language models, future token forecasting, hidden state representations, Bayesian inference, uncertainty quantification, visualization generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dBFDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper uses Bayesian linear classifiers on pre-generation hidden states to forecast distributions over visualization choices. It decomposes predictive uncertainty into aleatoric and epistemic terms. Across three LLMs, forecasts are stronger for broad design choices than field bindings, and epistemic uncertainty correlates with distributional error on several attributes.

Critical strengths

1. A useful focus on output distributions. Predicting distributions is appropriate when several chart designs are plausible. Distinguishing output variability from uncertainty in the forecaster is also a useful goal.

2. Substantial evaluation with useful controls. The reading/design/binding split exposes clear differences in difficulty. Source datasets are disjoint across splits. The supplement adds MLP and feature baselines, training-budget comparisons, and an exploratory comparison with human chart-effectiveness data.

Critical weaknesses

1. The added value of Bayesian uncertainty needs a clearer test. The MLP comparison changes model capacity, so it does not isolate the contribution of Bayesian inference. A matched linear baseline using maximum a posteriori (MAP) baseline and a direct comparison of epistemic uncertainty with predictive entropy would test whether the decomposition helps identify unreliable forecasts.

2. Some uncertainty interpretations need qualification. The paper acknowledges readout limitations, but the entropy gap alone does not separate missing representation information from limitations of the learned forecaster. The paper should present hidden-state information loss as a possible explanation for this gap throughout, as it does in the conclusion. Fig. 3(a) attributes entropy reduction from a uniform reference to hidden-state information, although class imbalance alone can reduce entropy. Class-frequency baselines appear elsewhere, but this interpretation still needs correction.

3. Inference details and computational costs are underreported. Appendix C omits posterior sample counts, warm-up, chain counts, and convergence diagnostics. This makes the uncertainty estimates harder to reproduce and assess. Reporting offline fitting costs separately from online prediction latency would also make the efficiency claim more concrete.

4. Reference comparisons would benefit from robustness checks. The supplement discloses filtering, uses entropy correction, and reports similar readout gaps on matched evaluation prompts. However, F1 scores and ceilings use different populations, with only 10-19 reference draws per eligible prompt. A larger-sample check using matched prompts and comparable decision rules would strengthen the evidence. Filtering also narrows the population covered by these results. This is a scope limitation, not a demonstrated source of error.

Questions for authors

1. What additional capability does the decomposition provide? Given prior work on behavior prediction, distribution recovery, and uncertainty probes [1-4], what added capability does this decomposition provide? Which results in the visualization setting establish that capability beyond what matched point-estimate probes and predictive entropy can provide?

2. How stable are the uncertainty estimates and reference comparisons? Please provide the posterior-sampling settings and convergence diagnostics. How sensitive are the readout gaps and F1 ceiling comparisons to the number of reference generations and to using matched evaluation prompts?

References

[1] Dong et al. Emergent Response Planning in LLMs. ICML, 2025.

[2] Ashok et al. Language Models Can Predict Their Own Behavior. NeurIPS, 2025.

[3] Piskorz et al. Eliciting Numerical Predictive Distributions of LLMs Without Auto-Regression. ICLR, 2026.

[4] Kossen et al. Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs. ICML FM-Wild Workshop, 2024.

Reviewer_PFhGOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Bayesian forecasting framework that predicts probability distributions over structured LLM output attributes using pre-generation hidden states, avoiding costly repeated generation. It employs Bayesian linear classifiers to estimate categorical output distributions and decomposes predictive uncertainty into aleatoric and epistemic components. Experiments on visualization specification generation across three LLMs demonstrate the predictability of different output attributes and investigate the relationship between uncertainty estimates and forecasting errors.

Critical strengths

The paper addresses a meaningful problem of efficiently estimating LLM output distributions without repeated generation. Using pre-generation hidden states provides a computationally attractive alternative to empirical sampling.

Critical weaknesses
  1. The proposed method mainly extends this setting to discrete structured output attributes using standard Bayesian classification and uncertainty decomposition. The paper does not sufficiently clarify or directly experimentally compare its methodological differences from these studies. For example, Ashok & May [1] and Piskorz et al. [2] have already investigated forecasting LLM outputs from pre-generation hidden states, including predicting output distributions without autoregressive generation.

  2. Bayesian uncertainty decomposition captures uncertainty over the forecaster's model parameters but cannot necessarily identify systematic prediction errors caused by model misspecification or insufficient information in hidden-state representations. In particular, low epistemic uncertainty does not guarantee accurate predictions, as the forecaster may produce consistently incorrect distributions even when its parameter uncertainty is low. Although the paper reports correlations between epistemic uncertainty and forecasting errors, these results do not sufficiently establish its reliability for detecting inaccurate forecasts.

[1] Ashok, D., & May, J. Language Models Can Predict Their Own Behavior. NeurIPS, 2025.

[2] Piskorz, J., Kobalczyk, K., & van der Schaar, M. Eliciting Numerical Predictive Distributions of LLMs Without Autoregression. arXiv:2603.02913, 2026.

Questions for authors
  1. What are the key methodological differences between the proposed framework and prior hidden-state forecasting approaches, particularly Ashok & May [1] and Piskorz et al. [2]? Could the authors provide comparisons with standard probabilistic classifiers and alternative uncertainty estimation methods to demonstrate the specific benefits of Bayesian forecasting?
  2. How reliable is epistemic uncertainty for identifying inaccurate forecasts when hidden-state representations lack relevant information, or the forecasting model is misspecified? Could the authors provide more direct evaluations, such as error detection or selective prediction experiments, to assess whether low epistemic uncertainty consistently corresponds to accurate output distribution estimates?

[1] Ashok, D., & May, J. Language Models Can Predict Their Own Behavior. NeurIPS, 2025.

[2] Piskorz, J., Kobalczyk, K., & van der Schaar, M. Eliciting Numerical Predictive Distributions of LLMs Without Autoregression. arXiv:2603.02913, 2026.

Reviewer_UbodOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLM hidden states can be used to predict their output distributions, where output distributions are marginal categorical distributions. Their method uses Bayesian logistic regression models with HMC posterior sampling, where predictive entropy is decomposed into expected conditional entropy (aleatoric uncertainty) and mutual information (epistemic uncertainty). Experiments focus on visualization with outputs constrained to a visual grammar, where the model is prompted with a dataset and goals the user wishes to analyze, and the LLM generates an appropriate visualization.

Critical strengths
  • Forecasting model output distributions from hidden activations is a timely and interesting topic.
  • Experimental results seem promising - that hidden states predict several kinds of behaviors and epistemic uncertainty correlates with forecasting error
Critical weaknesses
  • Visualization provides a useful controlled setting, but the authors' experiments are limited to one domain, which also requires outputs to be formatted according to a grammar. I would like to see some validation of these results with other tasks.
  • Claims about "readout gaps" seem a bit strong. The difference between the forecaster’s expected entropy and the empirical generator entropy could be an issue with the classifier or estimation error, rather than information absent from the hidden states.
  • I found Figures 2 and 3 to be somewhat complex and required some effort to parse. I wonder if either a different presentation or improved explanation would help.
  • Appendices seem to be missing from the paper .pdf that I see, meaning that many details of the work cannot be validated.
  • I would like to see some examples of the data that LLMs are being tested with and generating, but perhaps these were originally in the appendix.
Questions for authors
  • Is it possible to use/extend this method for domains where output does not necessarily match a structured format, such as a grammar?
  • Have you tested your method with more than 20 samples per prompt to verify its robustness?

Comments on the paper

2 comments
Reviewer_dBFDOpenReview ↗

Response to the Authors

Thank the authors for the clarifications. The response makes the intended role of epistemic uncertainty clearer. However, the following key concerns remain unresolved:

  • Q1/W1: Matched MAP and predictive-entropy comparisons are still needed to demonstrate the added value of Bayesian inference and uncertainty decomposition.
  • W2: The readout gap and entropy reduction from a uniform reference still require more cautious interpretation.
  • Q2/W3: Posterior-sampling settings, convergence diagnostics, and computational costs remain unreported.
  • Q2/W4: Further checks are needed to confirm that the results are stable with more reference samples, and that F1 scores and ceilings are evaluated on the same prompts using comparable prediction rules.

The main contents and appendix I have currently downloaded is identical to the initial versions, so I could not verify the proposed additions. Therefore, I tend to maintain my score.

We thank you again for your valuable feedback and concerns. We wanted to reply with our current findings before we decide to continue or withdraw current work.

Q1/W1: A direct comparison of MAP entropy's correlation to forecast error (in KL divergence) vs epistemic uncertainty's correlation to forecast error (also in KL) showed that MAP entropy was better correlated to forecast error. This result showed that we could not justify the added value of Bayesian inference which in essence, is a good measure for understanding model error.

We tried to identify the source of the problem, and currently our hypothesis is that epistemic uncertainty does not resolve well in the specific problem domain (visualization authoring). This is a critical finding that threatens the motivation for Bayesian inference in this setting.

Therefore, we are considering to withdraw the paper and properly reinforce our work for later opportunities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.