- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes reasoning-aware multicalibration, which incorporates chain-of-thought (CoT) properties into existing topic-based calibration for LLM reliability prediction. The authors provide a theoretical characterization of useful additional calibration groups and evaluate the framework on hallucination prediction and safety detection. Experiments show consistent Brier score improvements.
Critical strengths
- Well-motivated framework. The paper identifies systematic calibration errors overlooked by topic-only multicalibration and proposes a practical approach using observable CoT properties.
- Solid empirical evaluation. The experiments demonstrate consistent improvements across four QA benchmarks, with additional analyses of different annotators, calibration algorithms, cross-benchmark transfer, and safety prediction.
Critical weaknesses
- Limited methodological novelty. The framework primarily incorporates additional CoT-derived groups into an existing multicalibration algorithm. The theoretical result provides a necessary condition for improvement but does not establish when CoT groups will guarantee better performance.
- Insufficient comparison with alternative CoT-aware predictors. The paper does not directly compare against strong supervised calibration or reliability predictors that incorporate CoT features. Therefore, it remains unclear whether multicalibration provides a meaningful advantage over simpler approaches using the same information.
Questions for authors
- How does the proposed method compare with a supervised reliability predictor trained on the same initial scores, topic information, and CoT diagnostic features?
- Can the authors clarify the practical significance of theorem beyond characterizing when additional groups may be useful?
- How well does the framework generalize to reasoning traces generated by other LLM families, rather than only Qwen3-8B?
7 comments
Response to Weakness 1
W1: Limited methodological novelty and guaranteed-good method for group selection.
We kindly ask that the reviewer reads our official comment to all reviewers for our response about limited methodological novelty. As for the comment that "The theoretical result provides a necessary condition for improvement but does not establish when CoT groups will guarantee better performance," we add the following:
TLDR
A set of conditions that is guaranteed to select groups that decrease error is a hard open theoretical problem better suited for a theory paper. Instead, we are able to show a weaker statement: CoT contains information that would likely improve the multicalibration process. This alone is an interesting and novel result.
Extended Response
The reviewer highlights a very good open problem: Given a collection of potential additional groups, which ones, when included in multicalibration, are guaranteed to improve error? This is a restatement of our Definition 3.1. We do not answer this in our work as it is a very non-trivial theoretical problem whose answer is better suited for a primarily theoretical paper. We hope to present this problem to the multicalibration theory community by being the first work to formalize the group augmentation setting in Sec 3. To ensure it is clear that this problem is interesting future work, in the camera-ready, we will add it to a future work subsection in Sec 6: Discussion and Limitations. As for the case of guaranteed-good CoT groups specifically, note that here we actually do not even have pre-made groups, we must also feature engineer them. For this reason, we expect that the (weaker) conclusion that CoT very likely contains information to improve error is all that is reasonably possible. To see this, consider the complexity and variation of CoT reasoning, which is natural language data. The number of mappings of this natural language data to binary groups is unreasonably large. Thus, even if we had an answer to the general group selection question, in the case of CoT, the number of groups we must select from is extremely large. Assuming the complexity of the optimal group selection depends on the number of groups, it is likely that it would not be efficient to apply to CoT without first feature-engineering a smaller number of CoT-based groups to select from.
Response to Weakness 2
W2: Insufficient comparison with alternative CoT-aware predictors.
TLDR
The premise of work is that we must have an LLM reliability predictor that is guaranteed to be meaningful/trustworthy on all topic groups (while also getting low error). Multicalibration algorithms are (by definition) algorithms that achieve the guarantee. We do not want to acheive the lowest error possible if it is with a generic supervised algorithm that does not have the theoretical guarantee. Thus, comparison to generic algorithms is out of scope of the setting.
Extended Response
We divide this response into a three aspects of the weakness: (A) Comparing to other generic supervised learning algorithms, (B) Comparing to other multicalibration algorithms, ( C) Comparing to other simpler approaches.
A: Why don't we test other generic supervised learning algorithms? Multicalibration with respect to some collection of groups is a theoretical guarantee that a predictor be meaningful on each of those groups. More specifically, of all the times a predictor gives a probability of 99% for group A, 99% of those times, the binary outcome actually happens (Section 2, lines 140-153). The foundational premise of our work (lines 044-047) and also Detommaso et al, 2024 (see their Section 1) is that it is necessary this guarantee holds for topic groups when building a predictor of LLM reliablity. To see why this guarantee is good, consider, for example, the high-stakes topic of chemistry questions where recommendations from an LLM could lead to harmful experimentation by a user. It is important that in responses on this topic specifically, if the predictor says the output has 99% chance of safety, that this 99% a meaningful statistic (i.e., that genuinely for 99% of such responses, the output is safe). Having perfect accuracy (i.e. the Bayes optimal predictor) would obviously achieve this, but a generic algorithm that maximizes for accuracy or minimizes error is not guaranteed to achieve this (see lines 169-181), which is generally not possible. Thus, we focus on multicalibration algorithms rather than generic supervised learning algorithms. Of course, having lower error is still nice, so while retaining the meaningfulness guarantee on topic, we consider whether CoT (which has been shown to have signal for reliability, see Sec 1.1) can be usefully incorporated into the multicalibration process. Other supervised training methods definitely could achieve lower error (and even might achieve multicalibration, as we highlight in Q1!), but they do not explicitly optimize for it and thus are not guaranteed to achieve multicalibration in the way multicalibration algorithms are.
B: Why don't we test other multicalibration algorithms? We note that our framework is purposefully general and it can work with any multicalibration algorithm (see "the algorithm can be any out-of-the-box batch multicalibration algorithm" line 347). In the main body we conduct our case study using only IGHB, but in the appendix we also run the IGLB multicalibration algorithm (see E.1:Calibration algorithm and Table 7). With IGLB, we see similar trends in multicalibration w.r.t CoT and topic groups yielding lower error than topic-only.
C: The argument for the simplicity of multicalibration. As described in W2:B, our framework works for any generic multicalibration algorithm, but as the reviewer asks about "simpler approaches using the same information", we want to highlight that the patching-based multicalibration algorithms (IGHB and IGLB) we focus on are arguably the simpler than even many others that maximize accuracy and do not guarantee multicalibration. In particular, consider Algorithm 1. As written, it may appear complicated, but its only function is to iteratively modify the predictor's initial value for it's most uncalibrated sections such that the value over those spaces of is more calibrated. What this means is that if, for example, the initial function is already nearly multi-calibrated, the multicalibrated version will be nearly the same, but with the values for certain modified. As such, patching-based multicalibration algorithms carry a certain explainability and simplicity that much harder to achieve from a generic algorithm that takes in the initial function, topics, and CoT groupings and maximizes for accuracy. Without the structure of Algorithm 1, we cannot so simply describe the transformation of and if already carries some initial meaning, understanding how it has been changed is important.
We thank the reviewer for their notes of our well-motivated problem and strong empirical results. In this and the following comments, we respond to the weaknesses and questions listed in the review. For each weakness and question, we include a "too long; didn't read" (TLDR) response as well as an extended response. All responses are entirely human written. For parts of W1 we refer the reviewer to the comment we made for all reviewers. We are more than happy to engage further during this period and will try our best to respond to additional comments/questions.