- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for fine-tuning LLMs for calibrated decision-making over a discrete set of options. They propose a coherence loss to enforce consistent probabilities across related questions. Experiments are performed across the Decide-22 benchmark.
Critical strengths
Calibration is an important topic in reliable AI deployments. The community would benefit from more focus on LLMs as decision makers.
Critical weaknesses
I find that this paper does not distinguish its contributions from the extensive literature on LLM classification and calibration (e.g., see Section 8 in https://arxiv.org/abs/2602.08145 for extensive background). Text classification is not new; intent classification and toxicity detection have been prevalent for a long time in both research and industry. Fine-tuning for classification typically uses a proper scoring rule, cross-entropy. The paper begins with the claim “A growing share of calls to language models do not ask for text”; this should be cited or further substantiated.
The effects of post-training on calibration are well known (e.g., see https://arxiv.org/abs/2507.16806), and I am not convinced that this study adds substantially to that understanding.
Given the centering of the Decide-22 benchmark as a contribution of this work, it would be good to hear more motivation for why this collection of datasets is particularly meaningful.
The method is a combination of well-known techniques for fine-tuning and recalibrating LLM classifiers, plus a coherence penalty. The empirical effectiveness of the coherence penalty is not substantiated: (1) Table 1 shows no significant gains in accuracy or ECE (2) Comparisons are not made to obvious baselines, like using the additional data for fine-tuning.
Questions for authors
How is this setting different from typical classification with text inputs?
6 comments
"The effectiveness of the coherence penalty is not substantiated" (Reviewer BcEn):
Reviewer said: (1) Table 1 shows no significant gains in accuracy or ECE. (2) There's no comparison with obvious baselines, like using the additional data for fine-tuning.
"The effectiveness of the coherence penalty is not substantiated." On (1): the penalty isn't meant to raise accuracy or ECE. It targets incoherence, which it cuts tenfold (sure loss 0.291 → 0.024 at 9B), and Table 1 shows it does so at no cost in accuracy or calibration. On (2): we ran the baselines (details in our reply to yNja). Using the additional related questions as labelled training data also removes the incoherence (sure loss 0.034 at 9B), and so do a simple consistency loss (0.022) and the penalty (0.024). Four times as much ordinary labelled data does not (0.311). So the gain comes from training on the question families, not from the projection itself, and the revision says this plainly.
Novelty against classification and calibration work; post-training effects are known (Reviewer BcEn):
Reviewer said: ● The paper doesn't separate its contributions from the large literature on LLM classification and calibration (survey arXiv 2602.08145, Section 8). Text classification isn't new, and fine-tuning for classification already uses a proper scoring rule. ● The effects of post-training on calibration are well known (arXiv 2507.16806). ● The method is a combination of well-known techniques plus a coherence penalty. ● Question: how is this setting different from typical classification with text inputs?
How is this different from typical text classification? A text classifier has a fixed label set and is trained for one task. Here one model answers typed questions whose answer options arrive in the prompt at inference time: 22 tasks, three question types, and between 2 and 151 options. Six of the tasks are never used in fine-tuning, and accuracy on them still rises (74.5 → 79.9 at 9B). Most importantly, a classifier has no notion of coherence: whether its answers to logically related questions about the same text could all be true together (an option and the statement it implies, a statement and its negation, a score and a threshold). That property is what this paper is about. We agree that the fine-tuning loss itself is the standard one, cross-entropy as a proper scoring rule, and the paper doesn't claim it as new.
What the paper adds to the classification and calibration literature. The post-hoc methods in the survey you cite (Section 8.2) correct each answer on its own. Our findings are about answers across questions:
-
Decision fine-tuning of every kind we tried (on the readout, generative SFT, or with four times the data) makes a model's answers to related questions incoherent. At 9B, sure loss goes from 0.141 untuned to 0.23–0.31.
-
Recalibration can't fix it. We fitted temperature, Platt and contextual-calibration corrections directly on the test families to minimise incoherence, an optimistic bound. Any correction that keeps the model calibrated removes only 10–19% of the incoherence. Going further works only by flattening every answer, which raises ECE four- to six-fold and costs 3–9 points of accuracy.
-
Training on the question families removes it about tenfold (0.291 → 0.024 at 9B), on held-out tasks too, without costing accuracy or calibration. What incoherence is left now sits where people disagree: on ChaosNLI, the correlation between sure loss and human disagreement goes from −0.35 to +0.25 (Spearman, 9B), and sure loss starts to flag wrong answers (AUROC 0.44–0.49 → 0.57–0.71). Consistency training in general has prior work, which the paper cites (logic-rule losses, paraphrase-consistency losses, inference-time repair). What is new is showing that decision fine-tuning creates the problem, that recalibration can't remove it, and that family training does.
-
A benchmark that measures all of this (below).
"The effects of post-training on calibration are well known." That post-training hurts calibration is known, and the paper says so ("over-confidence after alignment is known"), citing this line of work: Kadavath et al. (2022) found that one temperature largely repairs an RLHF model's calibration, and Xiao et al. (2025) show ECE rising from SFT to DPO in Tülu 3. The revision also cites the paper you point to (Damani et al., 2025), which finds that RL with binary rewards hurts the confidence a model states after reasoning. Our question is a different one: what each stage of post-training does to a model's decisions. We measure the probabilities read in one forward pass, at every released stage of three open pipelines, with paired intervals, and this adds three things. The over-confidence enters at the preference step (ECE at T = 1 rises 0.03–0.08), and RLVR afterwards leaves it where it was (−0.009 and +0.010); this fits with Damani et al., since the confidence a model states after reasoning and the one-pass readout are different outputs. The change is almost a pure rescaling, so recalibration removes it and discrimination doesn't move. And readout fine-tuning undoes it: started from the SFT checkpoint or from the preference-optimized one, it lands in the same place. This sets up the paper's main result. The calibration damage from post-training can be undone by recalibration; the incoherence that decision fine-tuning creates cannot (recalibration removes only 10–19% of it), and training on question families removes it tenfold.
The paper's first sentence needs a source (Reviewer BcEn)
We agree the opening sentence needed a source, and it is much easier to support now than at submission. The revised first paragraph cites three documented uses of language models for closed-set decisions: labeling data (Gilardi et al., 2023), screening conversations for unsafe content (Inan et al., 2023), and judging other models' answers (Zheng et al., 2023). It also cites five products built specifically for such calls, each returning probabilities over a given set of answers rather than text, all released since September 2026: TypeSafe's Jev, Together AI's Tev1, OpenAI's Decisions API, Perplexity's Decisions API, and Google's Gemini Decisions API. Smaller companies have followed, for example, Liquid AI with its open-weight d1 models. OpenAI describes the uses as classifying content, routing requests, and prioritizing work, and Google uses its own as a query router inside its products. Since we did not measure a share of calls, the sentence now says "many calls" instead of "a growing share".