- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies selective abstention using an external verifier that scores a completed response without requiring access to the deployed generator’s weights or logits. DISCERN annotates responses with knowledge and inference chunks and produces chunk-level confidence scores. The verifier is trained with GRPO using a format gate and rewards for outcome detection, difficulty-aware mean scores, and confidence trajectories. Its central training mechanism is a sparring curriculum: a temporary solver is optimized to obtain high verifier scores, its regenerated responses are labeled using final-answer correctness, and the verifier continues training on these responses. Mean chunk confidence is the default response-selection score. The experiments train on DAPO-Math-17k and evaluate mathematical answer selection, reward and continuation controls, external ProcessBench traces, a larger verifier, and factual transfer. The strongest evidence concerns improved low-coverage mathematical selection on fixed solver responses; factual reliability remains substantially more limited.
Critical strengths
- The paper provides meaningful evidence of improved selection, beyond changes in confidence scale or generator accuracy. On frozen-base BeyondAIME responses, sparring improves mean-score AUC from 0.802 to 0.869, compared with 0.706 for static refresh. More importantly, the comparison against round 1 reports a 16.5-percentage-point reduction in expected risk at 20% coverage, with a paired interval excluding zero. Because the verifiers evaluate the same supplied responses within this comparison, and risk is compared at matched coverage, this is substantive evidence of better response selection—not merely a consequence of answering fewer questions or increasing the generator’s accuracy. This is the strongest consideration in favor of acceptance.
- The external, answer-supervised verification interface has useful empirical support beyond the original solver’s responses. The method does not require human step-error labels during training, yet the external-trace evaluations show useful error discrimination. For example, on the ProcessBench hard sets, DISCERN obtains higher whole-trace harmonic-mean scores than the off-the-shelf PRM under the specified transferred thresholds, while exhibiting a different error-recall/correct-pass trade-off. These results broaden the evidence beyond the initial solver and support the practical value of training an independently deployable verifier from answer-level supervision.
Critical weaknesses
- The central curriculum comparison needs stronger evidence that its advantage is robust to training and checkpoint selection. The paper does include the relevant fresh-response static control, and the improvement over round 1 is encouraging. However, both static-continuation configurations experienced declining validation reward, while the deployed sparring checkpoint was selected at a validation plateau. The manuscript does not make sufficiently explicit the validation split, selection criterion applied to each branch, or the training differences between the primary static-refresh and additional static-reference checkpoints. Consequently, it is difficult to assess how much of the gap reflects the response-generation strategy versus sensitivity to continuation settings or endpoint selection.
Furthermore, the five seeds are inference repetitions for fixed trained checkpoints, not independent training runs. The authors explicitly acknowledge that repeated paired continuation experiments from multiple round-1 initializations remain necessary. Given that the main curriculum result rests on 100 BeyondAIME problems, with smaller AIME sets, this limits confidence in the reproducibility of the main training contribution. My concern is not that the reported improvement is absent, but that its robustness as a training procedure is insufficiently established. - The benefit of the proposed scoring recipe over a comparably trained outcome-only verifier is not cleanly established. The single-reward control differs from DISCERN not only in reward terms, but also in format gating, sequence-length limits, and micro-batching. Moreover, the final-system comparison gives DISCERN an additional curriculum round relative to this control. These comparisons therefore establish differences between complete training recipes rather than isolate the contribution of the proposed reward design.
The component-removal experiments help, but their selection evidence is weaker: the full reward has the highest mean-score AUC point estimate, whereas minimum-score AUC favors removal variants, and the paired fixed-coverage risk intervals mostly include zero. The within-round outcome-only comparison primarily emphasizes SNR-Gain, whose dependence on score scale the paper correctly acknowledges. A matched outcome-only response-score baseline, evaluated using the deployed score and the same training budget, would be important for establishing the value of the additional machinery. The current results leave open whether much of the benefit comes from training a conventional correctness predictor on better data rather than from the particular chunk-score reward formulation.
Questions for authors
- What validation split and metric were used to select the round-1, sparring, primary static-refresh, and additional static-reference checkpoints? Was the same selection rule applied to all continuation branches? Please specify the training differences between the two static checkpoints, together with the solver-update budget and regenerated training-corpus sizes. Does the reported advantage persist when all branches are selected by the same held-out criterion?
- Under matched initialization, data, number of training rounds, format handling, and sequence-length limits, does the full reward outperform an outcome-only verifier on mean-score AUC, AURC, and matched-coverage risk? In particular, a control trained to predict a scalar response score—or directly optimize the deployed mean score—would help determine whether the proposed reward formulation improves selection beyond a simpler outcome predictor.
- What evidence is available that the sparring advantage over both round 1 and static refresh survives independent training randomness? Paired continuation repetitions from shared round-1 initializations would directly address this. If these are unavailable, a checkpoint-sensitivity analysis under a common validation-selection rule would still help determine whether the reported gap is stable or strongly endpoint-dependent.
4 comments
Rebuttal to Reviewer d4k6 (Part 2)
W3. Confounds in the Outcome-only comparison.
Reply: Thank you for pointing out the configuration issue. The differences described in the original manuscript were reporting errors. The original Outcome-only reward control and the first-round full-reward verifier were both initialized from Qwen3-4B-Instruct, used the same first-round DAPO training data, and were trained for 240 steps. Format gating, sequence length limit, micro-batching, and the structured output interface were also the same; only the reward function differed.
For comparison, Scalar does not segment and does not distinguish knowledge blocks from capability blocks; it directly gives a scalar score to the complete answer. Outcome-only uses only the outcome reward.
In the additional rebuttal experiments, Scalar, Outcome-only, and DISCERN use the same initial weights, training data, and per-step budget, and are compared at the same training step. For each baseline, we first select the checkpoint with the highest AUC averaged over five inference seeds among the evaluated checkpoints. We then compare it with DISCERN at the same training step. This setting is the most favorable to the baselines: each baseline is taken at its own best-AUC checkpoint, while DISCERN is simply read off at the same step rather than separately selected.
Evaluation follows the original 160-question setting, merging BeyondAIME, AIME 2024, and AIME 2025 and ranking them uniformly by score, with results averaged over five inference seeds. When ties occur at the truncation point, the expected value is used.
Table D: Comparison between DISCERN and Outcome-only at Outcome-only's best-AUC checkpoint.
| Configuration | Step | Mean AUC | Mean AURC | Risk@20% |
|---|---|---|---|---|
| DISCERN | 160 | 0.8621 | 0.4260 | 28.85% |
| Outcome-only | 160 | 0.8421 | 0.4390 | 28.86% |
As shown in Table D, at Outcome-only's best-AUC checkpoint (step 160), DISCERN achieves higher AUC and lower AURC, while Risk@20% is essentially the same.
W4. Component ablations and the need for a matched response-score baseline.
Reply: Thank you for this comment. The reward-component ablation results across five inference seeds are summarized in Table E.
Table E: One-round reward ablation across five inference seeds.
| One-round reward configuration | Mean AUC | Minimum AUC | Mean AURC |
|---|---|---|---|
| Full reward | 0.781 | 0.724 | 0.542 |
| Remove trajectory term | 0.760 | 0.753 | 0.565 |
| Remove difficulty term | 0.768 | 0.735 | 0.557 |
| Remove detection term | 0.729 | 0.729 | 0.578 |
Average segment confidence is used at deployment. As shown in Table E, when evaluated using this score, the full reward has the highest AUC and the lowest AURC.
Table F: Comparison between DISCERN and Scalar at Scalar's best-AUC checkpoint.
| Configuration | Step | Mean AUC | Mean AURC | Risk@20% |
|---|---|---|---|---|
| DISCERN | 240 | 0.8685 | 0.4174 | 27.69% |
| Scalar | 240 | 0.8222 | 0.4412 | 28.43% |
As shown in Table F, at Scalar's best-AUC checkpoint (step 240), DISCERN still achieves an AUC improvement of 0.0463, together with lower AURC and lower Risk@20%.
Q1. Validation split, selection rule, solver-update budgets, and regenerated corpus sizes.
Reply: Please see W1 of this response (Table A) for the unified checkpoint selection rule and the reselection results.
The two static branches use the same training configuration but start from independently trained round-1 checkpoints. Static reference was selected at step 80 using the same 443-question held-out set and AUC selection rule. The sparring checkpoint in the second experiment was also selected using the same held-out set and highest-AUC rule.
Both sparring solvers were updated for 67 steps. Regeneration covered 17,398 questions and produced 173,980 responses, of which 101,585 were retained for training.
Q2. Full reward vs. Outcome-only and scalar controls.
Reply: The matched comparisons show higher AUC and lower AURC for DISCERN than for Outcome-only and Scalar. The risk at matched coverage is essentially the same as Outcome-only and lower than Scalar. Please see W3 of this response (Table D) for the training configuration and Outcome-only results, and W4 (Table F) for the Scalar results.
Q3. Sparring across independent training runs.
Reply: The results show that sparring outperforms both the initial verifier and static continuation in both independent training experiments. Please see W2 of this response (Table C) for the paired continuation results.
In summary, we appreciate your constructive comments and suggestions. Please let us know if you have any other questions. We look forward to your feedback!
Thank you for the additional experiments. The common held-out checkpoint selection and paired continuation runs substantially address my main concerns. The matched scalar and outcome-only controls also strengthen the ranking results, although Risk@20% is essentially unchanged relative to Outcome-only. The evidence remains limited to two training experiments and small mathematical benchmarks. Within this scope, I support 3 weak acceptance.
Thank You and Final Clarifications
We thank the reviewer for the careful reassessment and for supporting acceptance.
We are glad that the common held-out checkpoint selection and paired continuation runs addressed the main concerns. We also agree that Risk@20% is essentially unchanged relative to Outcome-only, and we will state this explicitly rather than claim a risk improvement at that coverage level.
We acknowledge that our evidence is limited to two training experiments and small mathematical benchmarks. We will make this scope clear in the limitations section and phrase our conclusions accordingly, without extrapolating to other domains or larger-scale settings.
Thank you again for the constructive feedback.🙂
Rebuttal to Reviewer d4k6 (Part 1)
Thank you for your constructive comments and suggestions!
Reply: Following your suggestion, we specified a unified checkpoint selection rule before reselection and applied it to round 1, static refresh, and sparring. The held-out set contains 443 questions from the production environment, and it has been confirmed not to overlap with the BeyondAIME, AIME 2024, and AIME 2025 test questions. We compute ROC AUC using the average segment confidence, select the checkpoint with the highest AUC, and choose the earlier step in case of a tie. At test time, each verifier evaluates the same fixed set of solver answers, and results are averaged over inference seeds 42–46.
Table A: Checkpoint reselection under a unified held-out rule.
After reselecting checkpoints under the same rule, sparring's mean AUC on BeyondAIME is 0.8564, and static refresh is 0.7873. Compared with the reselected round 1, sparring reduces Risk@20% by 17.89 percentage points, with a paired interval of [10.01, 25.49] percentage points, which does not contain 0.
Reply: The Risk@20% reduction of sparring relative to round 1 is shown in Table B.
Table B: Risk@20% reduction of sparring relative to round 1.
To address your concern about independent training variability, we conducted two independent training experiments. In each experiment, we started from an independently trained round-1 checkpoint and performed paired static and sparring continuation training. The BeyondAIME mean AUC results are summarized in Table C.
Table C: BeyondAIME mean AUC in two independent training experiments.
The results show that sparring outperforms both round 1 and the corresponding static branch in both experiments.
The main comparison uses the first experiment, in which static performs better, in order to test the effect of sparring against the stronger static control. The second experiment was added during the rebuttal period with a limited number of training steps, and was started from a relatively weaker round-1 seed, so it shows the lower end of the gain.