- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for predicting the first failure in multi-turn and multi-step LLM conversations before it occurs. It formulates the problem using discrete-time survival analysis, incorporating current semantic uncertainty and historical instability to estimate first-failure risk. This work further applies conformal risk control to calibrate warning thresholds with a finite-sample guarantee on missed failures. Experiments on HalluHard and PRM800K show that incorporating historical information improves predictive performance and reduces warning burden compared with the evaluated baselines under a common missed-warning constraint.
Critical strengths
-
This paper studies an important question and distinguishes between detecting errors in a conversation and anticipating the first failure before the actual output is released, which is important for increasingly popular long-horizon LLM interactions.
-
The statistical formulation is coherent: survival analysis is a natural fit for this first-failure timing problem, and the paper carefully identifies and handles the risk set, censoring, and pre-response information. The conformal calibration is useful for translating a ranking score into an operational warning rule with a valid finite-sample guarantee on missed failures.
Critical weaknesses
-
The scope of the empirical evidence is limited. The HalluHard experiments use only two relatively small 3B target models and synthetic six-turn conversations. The first-failure labels are produced by larger same-family LLM judges. The judges are not validated against human factuality annotations and were checked only by having the same judge re-label 30 claims per setting (binary κ as low as 0.46). Also, the HalluHard analysis is conditioned on trajectories surviving until turn 3, which leaves small train/calibration/test splits in some settings (e.g., 24/8/8 split in Llama) and large variations between splits. The paper would benefit from experiments on more and stronger models, more natural interactions, and at least a meaningful human-validated subset of first-failure labels.
-
The evidence that the history feature H helps is limited. Figure 3 reports only means over 20 splits, without reporting error bars or significance tests, and the claimed gain is described as a ‘point-estimate’ improvement. Appendix A4 shows these gains are much smaller than the split-to-split variation. Also, in ablation results (Figure 4), the benefit of H is inconsistent, as column H falls below random AUROC for some settings (e.g., Qwen coding). As the authors noted, the contribution across survival models is “heterogeneous”. More discussion of this history feature H and clarification of its limitations is needed.
Questions for authors
-
Could the authors compare their geometrically weighted historical instability feature against a simple moving average of past (q) scores or directly including lagged (q) scores? This could help establish whether the geometric propagation construction itself matters rather than simply the availability of historical information.
-
The method requires K = 5 diagnostic generations before each monitored output. How does performance change with K (e.g., 3, 10, 20), and what is the extra inference cost and latency per turn?
-
The weakness I mentioned above.
4 comments
Q1 (geometric kernel). Thank you for this suggestion. We have added the comparison. Holding the hazard model, splits, and memory length fixed, we replaced with (1) the single lag , (2) the lags as separate covariates, with missing lags at early turns set to 0 and a missing-lag indicator for each lag (indicators collinear with the turn basis are dropped), and (3) a uniform moving average of past . The results separate the two explanations. Availability of history alone is not sufficient. The separate lags contain every input to , but they perform no better than the model without history (mean AUROC 0.548 versus 0.549). The likely reason is that estimating a separate coefficient for each lag from small training sets adds variance, whereas summarizes them with a single coefficient and one decay parameter. The specific geometric shape also matters little. The geometric kernel and the moving average perform comparably (0.588 versus 0.582), and both outperform the single lag (0.564). What matters is aggregating past instability into a single normalized summary. The geometric family covers both extremes ( recovers the single lag and the moving average), so the memory length is selected from data.
Table R5. Comparison of history summaries under the same hazard model. Each cell reports AUROC / AUPR on the landmark population (), averaged over 20 splits.
| Setting | No history | Single lag | Separate lags | Moving average | Geometric (ours) |
|---|---|---|---|---|---|
| Qwen Medical | 0.607 / 0.199 | 0.603 / 0.210 | 0.558 / 0.177 | 0.608 / 0.211 | 0.609 / 0.213 |
| Qwen Legal | 0.609 / 0.255 | 0.621 / 0.270 | 0.595 / 0.274 | 0.603 / 0.262 | 0.621 / 0.277 |
| Qwen Coding | 0.477 / 0.207 | 0.501 / 0.242 | 0.536 / 0.241 | 0.553 / 0.267 | 0.548 / 0.260 |
| Qwen Research | 0.494 / 0.102 | 0.521 / 0.123 | 0.504 / 0.112 | 0.488 / 0.110 | 0.513 / 0.124 |
| Llama Medical | 0.604 / 0.314 | 0.592 / 0.339 | 0.610 / 0.330 | 0.631 / 0.396 | 0.629 / 0.394 |
| Llama Legal | 0.482 / 0.441 | 0.488 / 0.475 | 0.474 / 0.437 | 0.548 / 0.482 | 0.570 / 0.503 |
| Llama Coding | 0.539 / 0.572 | 0.601 / 0.602 | 0.584 / 0.621 | 0.648 / 0.683 | 0.628 / 0.663 |
| Llama Research | 0.581 / 0.314 | 0.582 / 0.315 | 0.523 / 0.271 | 0.580 / 0.319 | 0.584 / 0.322 |
| Mean | 0.549 / 0.300 | 0.564 / 0.322 | 0.548 / 0.308 | 0.582 / 0.341 | 0.588 / 0.344 |
Q2 ( and cost). Due to cost and time constraints, we did not run or . However, we analyze how affects the cost. Each monitored turn requires extra responses from the same prefix, which can be generated in one batch with the released response, plus judge calls for semantic grouping (10 at ), which can run in parallel. Generation cost therefore grows linearly and grouping cost quadratically in . Larger would reduce the saturation of the uncertainty feature documented in Table A1, and we leave a full comparison to future work. The PRM800K setting requires no additional generation, since it uses existing sibling chains.
Q3. Please see our responses to W1 and W2 above.
Thank you for the detailed response and the additional analyses. My Q1 has been addressed. However, my concern about W2 is not fully addressed. The gain from H is not significant in any individual HalluHard setting, the pooled sign test shows a consistent direction rather than a meaningful effect size, and the coefficient on H is negative in many settings, which seems inconsistent with the paper’s motivation. I appreciate the authors acknowledging the limited scope in W1, but the concern remains unaddressed without additional evidence such as larger models or human-validated labels. I will therefore keep my score.
Thank you for the follow-up. We would like to clarify one point regarding the sign of the coefficient on H. Our claim is not that H is positively associated with first-failure risk, but that it carries risk information complementary to current-turn uncertainty. Our theoretical results are stated accordingly. In particular, Prop. B.2 holds regardless of sign, and Remark 1 in App. A.4 requires only a nonzero residual shift in H, in either direction. Section 5 (L417–424) also notes that the association of H with first failure is not uniformly positive. We also believe a negative coefficient is plausible, and the following conceptual example illustrates why.
Example Consider two patients monitored for heart disease whose current BMI is equally close to the alert threshold. The first is an athlete whose BMI has always been high, and the second has historically had a low BMI. If their current readings of BMI are identical, the second patient is more concerning, because the reading departs sharply from their own baseline. Likewise, given similar current uncertainty, a conversation that has been stable and then becomes uncertain can carry higher risk than one that has always been semantically diverse, which is what a negative coefficient on H captures.
Although we understand that the current manuscript may not have changed your assessment, we sincerely thank you for your time and constructive feedback.
W1 (scope). We agree that the empirical scope is limited. Our main goals are (1) a new formulation of early-warning monitoring for multi-turn dialogue and chain-of-thought reasoning, and (2) a first solution to this problem with a finite-sample guarantee. Accordingly, we used 3B target models to keep the cost of resampling diagnostic responses at every turn manageable, and the six-turn conversations follow the HalluHard protocol. Regarding labels, the PRM800K labels are human annotations, whereas the HalluHard labels come from LLM judges and were checked only for self-consistency. The small landmark splits make per-setting tests underpowered, which we address through tests across settings and the larger PRM800K evaluation (Table R3 in our response to Reviewer MY3o). Larger models, more natural interactions, and human-validated labels are important directions for future work.
W2 (evidence for H). Thank you for this comment. Overall, H provides a consistent but modest gain whose direction and size depend on the setting, and we will state this in the Discussion as its main limitation. We address each point below.