acceptodds
Under review as a conference paper at ICLR 2027

Dutch Books for Language Models

Abstract

People increasingly use language models to support life decisions. Many such decisions involve a probabilistic forecast: How likely is a major life event, a natural disaster, or an economic outcome? Users of language models may implicitly trust that these forecasts fall out of a coherent world model. In this paper, we evaluate the coherence of language model probabilistic forecasts through a procedure that builds on a theorem due to de Finetti. We elicit forecasts from language models across events generated from stock returns data. We then use linear programs to compute the largest Dutch-book profit - the profit an arbitrageur could guarantee by betting against model-generated probabilities - which we use as a measure of incoherence. Our procedure does not require outcome labels, so we can evaluate coherence even in settings where outcomes are not observed or have not yet resolved. We find substantial evidence of incoherence in language model forecasts. Such incoherence increases when there are richer logical relationships between events, and irrelevant contextual details can increase incoherence by an order of magnitude. We conclude by discussing how alternative training strategies may improve probabilistic coherence.

Primary area:
foundation or frontier models, including LLMs
Keywords:
probabilistic coherence, evaluation, forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wzdqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the coherence of LLM probability forecasts through a bounded-stake Dutch-book linear program. Its optimal value equals the minimum sup-norm distance to the set of coherent forecasts, enabling evaluation without realized outcomes. Experiments cover 15 models and 100 stock-days, with detailed interventions conducted on GPT-OSS-120B. The results document substantial variation across models, additional inconsistencies in joint-event forecasts, and strong dependence on elicitation protocol. Full-panel elicitation nearly eliminates the workhorse’s incoherence, while some contextual additions substantially increase it.

Critical strengths
  1. An exact diagnostic with an interpretable certificate. For the specified event space, the LP determines whether the reported probabilities admit a common distribution and produces a portfolio witnessing any incompatibility. Its dual quantifies the smallest maximum probability adjustment needed to restore coherence. This is a useful evaluation criterion when outcomes remain unresolved.

  2. Informative elicitation comparisons with relevant controls. The study includes date-clustered uncertainty estimates, repeated elicitations, complementary and less-related question groupings, and fixed-size linked/dispersed panels. The contrast between isolated and full-panel elicitation demonstrates that measured coherence can depend substantially on how related forecasts are requested.

Critical weaknesses
  1. The paper does not establish the empirical value of its principal methodological distinction. Paleka et al. (2025) already formulate consistency through worst-case arbitrage, although with a different scoring geometry, and Andrews (2026) supplies the bounded-stake formulation used here. The contribution therefore rests on what exhaustive evaluation reveals empirically. That case is missing. In the one-day setting, all 14 probabilities are determined by four bin masses with only three degrees of freedom; normalization and consistency of unions with those masses already characterize coherence. Figure 1 illustrates an ordinary three-event additivity failure. The paper does not show whether the full LP reveals consequential failures beyond local checks in the richer settings, or whether its additional coverage changes any substantive conclusion.

  2. The strongest findings remain tied to an unexplained, unusually coherent baseline. All detailed interventions concern GPT-OSS-120B, selected partly for its favorable preliminary coherence. The study does not establish whether its baseline forecasts largely reproduce empirical bin frequencies, which are coherent by construction, or whether the interventions cause inconsistent departures from that procedure. This distinction matters for interpreting what the experiments teach about probabilistic forecasting. The baseline comparison across 15 models does not resolve the issue because those models are not evaluated under the same interventions. Moreover, the approximately 100-fold range in incoherence provides limited evidence of diagnostic superiority by itself: raw Brier scores include outcome uncertainty, so their relative spread is not directly comparable.

Questions for authors
  1. Under the same total-stake normalization, how much additional profit does the full LP identify beyond portfolios restricted to complement, additivity, and marginalization relations?
Reviewer_h6fNOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper examines probabilistic coherence in a set of language model forecasts of stock data by measuring whether they indicate the existence of a strategy which yields a profit regardless of outcome. Various models are shown to produce incoherent predictions, and incoherence is shown to vary more than forecast error. Analysis of joint events and forecast horizons show increased incoherence due to logical dependencies, as do various prompt changes. However, providing statistics of the binned historical returns or asking about a full panel of events in a single query drives incoherence near zero, indicating that incoherence in other experiments is related to the event elicitation strategy (which the authors assume is likely to be practiced by general users). They recommend post-training models to explicitly reason about related or logically dependent events.

Critical strengths

Task, experiments, and results are clearly explained. High level motivation about coherent world models is interesting. Financial forecasting is an operationalization for testing world model coherence.

Critical weaknesses

It is shown that electing a set of independent forecasts for logically dependent events leads to incoherence. This does not demonstrate the model’s internal “world model” is incoherent, but only that independent sampling is incoherent. In fact, the coherence of joint forecasts demonstrates that models can produce outputs which could obtain in coherent world models.

The authors recommend post training a model to reason about dependent events before answering. This may be possible for the specific forecasting task, but in a more general case it is extremely challenging to identify what events are logically dependent on each other. Also, it’s not clear that we want models from which we can draw independent samples that display logical coherence with each other: uncertainty (which is present in the independent output distributions but not samples from that distribution) is a desirable characteristic in many situations even in the financial domain.

The paper argues that “general users will query models for forecasts” with an independent elicitation strategy rather than jointly. This is an unsupported assertion. Moreover, general users should not be querying models for financial forecasts with any strategy. Models should refuse these prompts for the safety of the general user.

A prompt which includes binned statistics increases coherence, indicating that incoherence issues might be related to the mismatch between the input and required output format rather than an incoherent world model.

Questions for authors

While the high-level motivation around world models is interesting, is it necessary? This paper seems to use the world model motivation when convenient but retreats to the domain specific problem setting otherwise (noted in the weaknesses).

Reviewer_41UZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper measures the probabilistic coherence of forecasts from language models using de Finetti’s Dutch book theorem. The authors get probabilities for events whose logical structure is known: i.e. variance normalized next day stock returns falling in four bins, plus unions, complements, and joint events across days and assets. Then a linear program calculates the largest guaranteed arbitrage profit per unit of gross stake. By LP duality this equals the sup norm distance from the stated probabilities to the set of coherent forecasts. The measure needs no outcome labels. The study covers 100 stock days, 15 models, and GPT-OSS 120B as the main model. The authors report the following:

  • Incoherence is present in all models and varies around 100x across them, whereas Brier scores vary much less.
  • The elicitation format matters a lot: forecasting instructions, company identity, and irrelevant “red herring” context all raise incoherence. Asking for the full event panel in one query pretty much removes it.
  • Averaging repeated elicitations helps a little bit.
  • The method also applies to unresolved long-horizon events.
Critical strengths
  • The metric itself is well thought out and easy to understand: It can be understood in two ways that are the same: as the profit someone could guarantee by betting against the model’s probabilities or as how much the probabilities would need to be adjusted to be consistent.
  • Because it is label free, it applies to unresolved or never resolving questions, which is shown in appendix B.
  • The domain choice is well chosen. Logical relations are known fully and the sample window comes after the main model’s release to limit lookahead bias.
  • The confidence intervals use a bootstrap clustered by date, and the additional context arms use paired comparison, which are good practices.
Critical weaknesses
  1. In the baseline GPT-OSS-120B has ℓ∞ around ~ 0.0021. This means its stated probabilities are within 0.2 percentage points of a coherent forecast in sup norm. That is hard to call as “substantial evidence of incoherence” and the abstract’s wording around life decisions. The larger values for other models and for some elicitation arms (roughly 0.02-0.2) are more convincing evidence and could carry more of the argument.

  2. Normalization by gross stake: The ℓ1 bound on stakes dilutes violations that involve many events. In Figure 1 an additivity violation of about 0.033 gives ℓ∞ ~ 0.011 because three events share the correction. Constraints from joint and multi day panels usually involve more events. Therefore, comparisons across panel structures mix the size of a violation with the number of events in the binding constraint.

  3. Section 4.2 could have some changes:

  • “Varies by 100x versus a narrow range” compares a measure with floor zero to one with an irreducible floor around 0.19. Ratios of the two are not exactly comparable. A Brier skill score against a reference forecaster would be more meaningful, for example one that uses historical bin frequencies.
  • Part of the correlation is mechanical: For the Brier score, an incoherent forecast is overshadowed by its projection onto the coherent set. Reporting the Brier score of the projected forecasts would show whether models still differ in accuracy once incoherence is removed.
  • Counting significant pairwise differences (101 for coherence vs 32 for Brier) compares metrics with very different noise structures. With 100 stock days, Brier is limited by outcome noise.
  1. Confounds in the cross model comparison: Each provider used its default reasoning setting, temperature was not set for GPT-5.6 or Claude, and each non workhorse model got one pass. The coherence ranking may partly track reasoning budget. Reporting reasoning effort or tokens would help, as would checking whether the ranking holds under one or two other elicitation arms. The limitations section acknowledges this, but it limits what Figure 2 can show.
Questions for authors
  1. What does “95 percent of answers are identical across all five passes” mean exactly: per query? At temperature 1 with a different seed per pass, this is surprising. Can you confirm that sampling on the provider was not effectively deterministic?
  2. Do retries on refusals or erroneous/malformed outputs select for a particular kind of response?
  3. How are Brier scores aggregated over the 14 overlapping events, which are highly dependent?
  4. Were multiple-comparison adjustments considered across the many elicitation arms?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.