acceptodds
Under review as a conference paper at ICLR 2027

DISCERN: Decoupling Reasoning and Verification for Reliable Abstention

Abstract

An external verifier can help a language model abstain from incorrect answers, but its training responses determine which errors it encounters. Can the verifier itself guide the generation of more useful training responses? We propose DISCERN, an answer-supervised chunk-scoring verifier trained through a sparring curriculum. A temporary solver learns to obtain high scores from the current verifier; final-answer labels then distinguish correct responses from errors that received high scores, providing supervision for the next verifier update. Deployment requires no generator weights or logits. On 100 greedy BeyondAIME responses, the final 4B verifier achieves mean-score AUC 0.913. In the continuation comparison on frozen-base BeyondAIME responses, sparring raises mean-score AUC from 0.802 to 0.869, compared with 0.706 for static refresh with an unchanged solver, and lowers expected risk relative to round 1 by 16.5 percentage points at 20% coverage. Same-trace ProcessBench comparisons extend evaluation to GSM8K, MATH, OlympiadBench, and Omni-MATH. Tests with a 30B verifier and external generators broaden the evaluation, while SimpleQA remains difficult. The clearest gains are in mathematical response selection, particularly when accepting only a small fraction of responses.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
large language models, selective prediction, abstention, verifier training, mathematical reasoning, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_d4k6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies selective abstention using an external verifier that scores a completed response without requiring access to the deployed generator’s weights or logits. DISCERN annotates responses with knowledge and inference chunks and produces chunk-level confidence scores. The verifier is trained with GRPO using a format gate and rewards for outcome detection, difficulty-aware mean scores, and confidence trajectories. Its central training mechanism is a sparring curriculum: a temporary solver is optimized to obtain high verifier scores, its regenerated responses are labeled using final-answer correctness, and the verifier continues training on these responses. Mean chunk confidence is the default response-selection score. The experiments train on DAPO-Math-17k and evaluate mathematical answer selection, reward and continuation controls, external ProcessBench traces, a larger verifier, and factual transfer. The strongest evidence concerns improved low-coverage mathematical selection on fixed solver responses; factual reliability remains substantially more limited.

Critical strengths
  1. The paper provides meaningful evidence of improved selection, beyond changes in confidence scale or generator accuracy. On frozen-base BeyondAIME responses, sparring improves mean-score AUC from 0.802 to 0.869, compared with 0.706 for static refresh. More importantly, the comparison against round 1 reports a 16.5-percentage-point reduction in expected risk at 20% coverage, with a paired interval excluding zero. Because the verifiers evaluate the same supplied responses within this comparison, and risk is compared at matched coverage, this is substantive evidence of better response selection—not merely a consequence of answering fewer questions or increasing the generator’s accuracy. This is the strongest consideration in favor of acceptance.
  2. The external, answer-supervised verification interface has useful empirical support beyond the original solver’s responses. The method does not require human step-error labels during training, yet the external-trace evaluations show useful error discrimination. For example, on the ProcessBench hard sets, DISCERN obtains higher whole-trace harmonic-mean scores than the off-the-shelf PRM under the specified transferred thresholds, while exhibiting a different error-recall/correct-pass trade-off. These results broaden the evidence beyond the initial solver and support the practical value of training an independently deployable verifier from answer-level supervision.
Critical weaknesses
  1. The central curriculum comparison needs stronger evidence that its advantage is robust to training and checkpoint selection. The paper does include the relevant fresh-response static control, and the improvement over round 1 is encouraging. However, both static-continuation configurations experienced declining validation reward, while the deployed sparring checkpoint was selected at a validation plateau. The manuscript does not make sufficiently explicit the validation split, selection criterion applied to each branch, or the training differences between the primary static-refresh and additional static-reference checkpoints. Consequently, it is difficult to assess how much of the gap reflects the response-generation strategy versus sensitivity to continuation settings or endpoint selection.
    Furthermore, the five seeds are inference repetitions for fixed trained checkpoints, not independent training runs. The authors explicitly acknowledge that repeated paired continuation experiments from multiple round-1 initializations remain necessary. Given that the main curriculum result rests on 100 BeyondAIME problems, with smaller AIME sets, this limits confidence in the reproducibility of the main training contribution. My concern is not that the reported improvement is absent, but that its robustness as a training procedure is insufficiently established.
  2. The benefit of the proposed scoring recipe over a comparably trained outcome-only verifier is not cleanly established. The single-reward control differs from DISCERN not only in reward terms, but also in format gating, sequence-length limits, and micro-batching. Moreover, the final-system comparison gives DISCERN an additional curriculum round relative to this control. These comparisons therefore establish differences between complete training recipes rather than isolate the contribution of the proposed reward design.
    The component-removal experiments help, but their selection evidence is weaker: the full reward has the highest mean-score AUC point estimate, whereas minimum-score AUC favors removal variants, and the paired fixed-coverage risk intervals mostly include zero. The within-round outcome-only comparison primarily emphasizes SNR-Gain, whose dependence on score scale the paper correctly acknowledges. A matched outcome-only response-score baseline, evaluated using the deployed score and the same training budget, would be important for establishing the value of the additional machinery. The current results leave open whether much of the benefit comes from training a conventional correctness predictor on better data rather than from the particular chunk-score reward formulation.
Questions for authors
  1. What validation split and metric were used to select the round-1, sparring, primary static-refresh, and additional static-reference checkpoints? Was the same selection rule applied to all continuation branches? Please specify the training differences between the two static checkpoints, together with the solver-update budget and regenerated training-corpus sizes. Does the reported advantage persist when all branches are selected by the same held-out criterion?
  2. Under matched initialization, data, number of training rounds, format handling, and sequence-length limits, does the full reward outperform an outcome-only verifier on mean-score AUC, AURC, and matched-coverage risk? In particular, a control trained to predict a scalar response score—or directly optimize the deployed mean score—would help determine whether the proposed reward formulation improves selection beyond a simpler outcome predictor.
  3. What evidence is available that the sparring advantage over both round 1 and static refresh survives independent training randomness? Paired continuation repetitions from shared round-1 initializations would directly address this. If these are unavailable, a checkpoint-sensitivity analysis under a common validation-selection rule would still help determine whether the reported gap is stable or strongly endpoint-dependent.
4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer d4k6 (Part 1)

Thank you for your constructive comments and suggestions!

W1. Robustness to checkpoint selection.

Reply: Following your suggestion, we specified a unified checkpoint selection rule before reselection and applied it to round 1, static refresh, and sparring. The held-out set contains 443 questions from the production environment, and it has been confirmed not to overlap with the BeyondAIME, AIME 2024, and AIME 2025 test questions. We compute ROC AUC using the average segment confidence, select the checkpoint with the highest AUC, and choose the earlier step in case of a tie. At test time, each verifier evaluates the same fixed set of solver answers, and results are averaged over inference seeds 42–46.

Table A: Checkpoint reselection under a unified held-out rule.

BranchStepValidation mean AUCBeyondAIME mean AUCAURCRisk@20%
Round 12400.74380.78880.577248.43%
Static refresh3200.77580.78730.574746.52%
Sparring2800.77860.85640.493630.54%

After reselecting checkpoints under the same rule, sparring's mean AUC on BeyondAIME is 0.8564, and static refresh is 0.7873. Compared with the reselected round 1, sparring reduces Risk@20% by 17.89 percentage points, with a paired interval of [10.01, 25.49] percentage points, which does not contain 0.

W2. Training reproducibility beyond inference seeds.

Reply: The Risk@20% reduction of sparring relative to round 1 is shown in Table B.

Table B: Risk@20% reduction of sparring relative to round 1.

DatasetRisk@20% reduction (percentage points)Paired interval (percentage points)
BeyondAIME17.89[10.01, 25.49]
AIME 20244.27[0.37, 8.89]
AIME 20256.20[-1.51, 17.28]

To address your concern about independent training variability, we conducted two independent training experiments. In each experiment, we started from an independently trained round-1 checkpoint and performed paired static and sparring continuation training. The BeyondAIME mean AUC results are summarized in Table C.

Table C: BeyondAIME mean AUC in two independent training experiments.

Independent training experimentRound-1 mean AUCStatic branchStatic mean AUCSparring mean AUC
First group (main comparison)0.7888Static refresh0.78730.8564
Second group0.7280Static reference0.77440.7877

The results show that sparring outperforms both round 1 and the corresponding static branch in both experiments.

The main comparison uses the first experiment, in which static performs better, in order to test the effect of sparring against the stronger static control. The second experiment was added during the rebuttal period with a limited number of training steps, and was started from a relatively weaker round-1 seed, so it shows the lower end of the gain.

AuthorsOpenReview ↗

Rebuttal to Reviewer d4k6 (Part 2)

W3. Confounds in the Outcome-only comparison.

Reply: Thank you for pointing out the configuration issue. The differences described in the original manuscript were reporting errors. The original Outcome-only reward control and the first-round full-reward verifier were both initialized from Qwen3-4B-Instruct, used the same first-round DAPO training data, and were trained for 240 steps. Format gating, sequence length limit, micro-batching, and the structured output interface were also the same; only the reward function differed.

For comparison, Scalar does not segment and does not distinguish knowledge blocks from capability blocks; it directly gives a scalar score to the complete answer. Outcome-only uses only the outcome reward.

In the additional rebuttal experiments, Scalar, Outcome-only, and DISCERN use the same initial weights, training data, and per-step budget, and are compared at the same training step. For each baseline, we first select the checkpoint with the highest AUC averaged over five inference seeds among the evaluated checkpoints. We then compare it with DISCERN at the same training step. This setting is the most favorable to the baselines: each baseline is taken at its own best-AUC checkpoint, while DISCERN is simply read off at the same step rather than separately selected.

Evaluation follows the original 160-question setting, merging BeyondAIME, AIME 2024, and AIME 2025 and ranking them uniformly by score, with results averaged over five inference seeds. When ties occur at the truncation point, the expected value is used.

Table D: Comparison between DISCERN and Outcome-only at Outcome-only's best-AUC checkpoint.

ConfigurationStepMean AUCMean AURCRisk@20%
DISCERN1600.86210.426028.85%
Outcome-only1600.84210.439028.86%

As shown in Table D, at Outcome-only's best-AUC checkpoint (step 160), DISCERN achieves higher AUC and lower AURC, while Risk@20% is essentially the same.

W4. Component ablations and the need for a matched response-score baseline.

Reply: Thank you for this comment. The reward-component ablation results across five inference seeds are summarized in Table E.

Table E: One-round reward ablation across five inference seeds.

One-round reward configurationMean AUCMinimum AUCMean AURC
Full reward0.7810.7240.542
Remove trajectory term0.7600.7530.565
Remove difficulty term0.7680.7350.557
Remove detection term0.7290.7290.578

Average segment confidence is used at deployment. As shown in Table E, when evaluated using this score, the full reward has the highest AUC and the lowest AURC.

Table F: Comparison between DISCERN and Scalar at Scalar's best-AUC checkpoint.

ConfigurationStepMean AUCMean AURCRisk@20%
DISCERN2400.86850.417427.69%
Scalar2400.82220.441228.43%

As shown in Table F, at Scalar's best-AUC checkpoint (step 240), DISCERN still achieves an AUC improvement of 0.0463, together with lower AURC and lower Risk@20%.

Q1. Validation split, selection rule, solver-update budgets, and regenerated corpus sizes.

Reply: Please see W1 of this response (Table A) for the unified checkpoint selection rule and the reselection results.

The two static branches use the same training configuration but start from independently trained round-1 checkpoints. Static reference was selected at step 80 using the same 443-question held-out set and AUC selection rule. The sparring checkpoint in the second experiment was also selected using the same held-out set and highest-AUC rule.

Both sparring solvers were updated for 67 steps. Regeneration covered 17,398 questions and produced 173,980 responses, of which 101,585 were retained for training.

Q2. Full reward vs. Outcome-only and scalar controls.

Reply: The matched comparisons show higher AUC and lower AURC for DISCERN than for Outcome-only and Scalar. The risk at matched coverage is essentially the same as Outcome-only and lower than Scalar. Please see W3 of this response (Table D) for the training configuration and Outcome-only results, and W4 (Table F) for the Scalar results.

Q3. Sparring across independent training runs.

Reply: The results show that sparring outperforms both the initial verifier and static continuation in both independent training experiments. Please see W2 of this response (Table C) for the paired continuation results.

In summary, we appreciate your constructive comments and suggestions. Please let us know if you have any other questions. We look forward to your feedback!

Thank you for the additional experiments. The common held-out checkpoint selection and paired continuation runs substantially address my main concerns. The matched scalar and outcome-only controls also strengthen the ranking results, although Risk@20% is essentially unchanged relative to Outcome-only. The evidence remains limited to two training experiments and small mathematical benchmarks. Within this scope, I support 3 weak acceptance.

AuthorsOpenReview ↗

Thank You and Final Clarifications

We thank the reviewer for the careful reassessment and for supporting acceptance.

We are glad that the common held-out checkpoint selection and paired continuation runs addressed the main concerns. We also agree that Risk@20% is essentially unchanged relative to Outcome-only, and we will state this explicitly rather than claim a risk improvement at that coverage level.

We acknowledge that our evidence is limited to two training experiments and small mathematical benchmarks. We will make this scope clear in the limitations section and phrase our conclusions accordingly, without extrapolating to other domains or larger-scale settings.

Thank you again for the constructive feedback.🙂

Reviewer_aCHUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether an external verifier can improve its own training data by guiding a temporary solver. DISCERN scores knowledge and inference chunks in a supplied response, then aggregates those scores to decide whether to accept the answer. Training alternates between updating a solver to obtain high verifier scores and updating the verifier on regenerated responses labeled by final-answer correctness. The verifier combines error-detection, difficulty-aware, and score-trajectory rewards, without requiring step-level training labels.

The strongest results concern mathematical answer selection. On fixed BeyondAIME responses, sparring improves mean-score AUC from 0.802 to 0.869, versus 0.706 for static refresh, and reduces expected risk by 16.5 percentage points at 20% coverage relative to round 1. Additional experiments evaluate reward components, externally generated ProcessBench traces, a larger verifier, and generator transfer. Factual transfer remains substantially weaker.

Critical strengths

The sparring curriculum addresses a meaningful training problem with a sensible supervision boundary. Optimizing the solver against verifier scores can expose confidently misjudged responses, while independent final-answer labels prevent those responses from automatically becoming positive examples. Table 4 evaluates both continuation branches on fixed response populations. The improvement over round 1, together with the low-coverage risk reductions in Table 2, provides evidence beyond simply outperforming a weak static-refresh endpoint.

The evaluation includes useful checks beyond the original solver’s answers. Same-trace ProcessBench comparisons test whether the verifier transfers to externally supplied mathematical reasoning. DISCERN improves whole-trace rejection F1 over the PRM on OlympiadBench and Omni-MATH, although localization is approximately tied. I also appreciate the explicit separation of ranking, calibration, and SNR-Gain, and the candid reporting of poor absolute precision on SimpleQA.

Critical weaknesses

The central curriculum comparison does not yet establish a robust advantage over well-selected static continuation. Static refresh reduces frozen-base BeyondAIME AUC from 0.802 to 0.706, and Appendix E.4 reports declining validation reward in both static runs. This makes the checkpoint-selection procedure consequential: comparison against a deteriorating endpoint may overstate the benefit relative to a validation-selected static model. Five inference seeds quantify variability for fixed trained checkpoints, not variability in learning the curriculum. The improvement over round 1 is encouraging, but its clearest matched-coverage evidence comes from a 100-problem benchmark; the reported 20%-coverage intervals on both AIME sets include zero. These limitations leave uncertainty about the repeatability of the main contribution.

The benefit of the structured reward and chunk decomposition is only partially isolated. The outcome-only control differs in format gating, sequence limits, and micro-batching (Appendix M.1), while the final-system comparison additionally includes another training round and regenerated data. Component removals provide cleaner evidence, but their mean-AUC gains are modest and most paired fixed-coverage risk intervals include zero. There is also no matched scalar verifier or fixed-segmentation control. Consequently, the results establish that the complete system works better than several reported alternatives, but do not convincingly establish how much the chunk structure and trajectory preferences contribute. The 83.0% semantic-tag agreement measures readability rather than their value for abstention.

Questions for authors

How were the primary static-refresh, additional static-reference, and sparring checkpoints selected? What validation data and metric were used, what training settings distinguish the two static runs, and how does sparring compare with the best validation-selected static checkpoint?

Does the continuation advantage persist when both branches are repeated from independently trained round-1 checkpoints? Even a limited repeat of this central comparison would help distinguish a curriculum effect from training-run variability.

Does the full reward retain its ranking and matched-coverage risk advantage when the outcome-only control uses identical format gating, sequence limits, micro-batching, training data, and update budget? This would materially strengthen the evidence for the reward recipe.

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer aCHU (Part 1)

Thank you for your constructive comments and suggestions!

W1. Dependence on checkpoint selection, training variability, and statistical significance.

Reply: Thank you for this comment. Following your suggestion, we have reselected the checkpoints for round 1, static refresh, and sparring according to a unified held-out set selection rule (see Table E in Q1). After reselection, sparring's mean AUC on BeyondAIME is 0.8564, and static refresh is 0.7873.

Compared with the reselected round 1, the Risk@20% reduction of sparring is shown in Table A.

Table A: Risk@20% reduction of sparring relative to round 1.

DatasetRisk@20% reduction (percentage points)Paired interval (percentage points)
BeyondAIME17.89[10.01, 25.49]
AIME 20244.27[0.37, 8.89]
AIME 20256.20[-1.51, 17.28]

W2. Isolation of reward and chunking effects.

Reply: Thank you for pointing out the configuration issue. The differences described in the original manuscript were reporting errors. We would like to clarify that the original Outcome-only reward control and the first-round full-reward verifier were both initialized from Qwen3-4B-Instruct, used the same first-round DAPO training data, and were trained for 240 steps. Both sampled 8 rollouts per question, used a training batch size of 256 and a learning rate of 10−610^{-6}. Format gating, sequence length limit, micro-batching, and the structured output interface were also the same; only the reward function differed.

At deployment, average segment confidence is used for filtering. The one-round reward ablation results are summarized in Table B, with all results averaged over five inference seeds.

Table B: One-round reward ablation.

One-round reward configurationMean AUCMean AURC
Full reward0.7810.542
Remove trajectory term0.7600.565
Remove difficulty term0.7680.557
Remove detection term0.7290.578

For comparison, Scalar does not segment and does not distinguish knowledge blocks from capability blocks; it directly gives a scalar score to the complete answer. Outcome-only uses only the outcome reward.

In the additional rebuttal experiments, Scalar, Outcome-only, and DISCERN use the same initial weights, training data, and per-step budget, and are compared at the same training step. For each baseline, we first select the checkpoint with the highest AUC averaged over five inference seeds among the evaluated checkpoints. We then compare it with DISCERN at the same training step. This setting is the most favorable to the baselines: each baseline is taken at its own best-AUC checkpoint, while DISCERN is simply read off at the same step rather than separately selected.

Evaluation follows the original 160-question setting, merging BeyondAIME, AIME 2024, and AIME 2025 and ranking them uniformly by score, with results averaged over five inference seeds.

Table C: Comparison between DISCERN and Outcome-only at Outcome-only's best-AUC checkpoint.

ConfigurationStepMean AUCMean AURCRisk@20%
DISCERN1600.86210.426028.85%
Outcome-only1600.84210.439028.86%

As shown in Table C, at Outcome-only's best-AUC checkpoint (step 160), DISCERN achieves higher AUC and lower AURC, while Risk@20% is essentially the same.

Table D: Comparison between DISCERN and Scalar at Scalar's best-AUC checkpoint.

ConfigurationStepMean AUCMean AURCRisk@20%
DISCERN2400.86850.417427.69%
Scalar2400.82220.441228.43%

As shown in Table D, at Scalar's best-AUC checkpoint (step 240), DISCERN still achieves higher AUC, lower AURC, and lower Risk@20%.

AuthorsOpenReview ↗

Rebuttal to Reviewer aCHU (Part 2)

Q1. Checkpoint selection for static-refresh, static-reference, and sparring runs.

Reply: Following your suggestion, we specified a unified checkpoint selection rule before reselection and applied it to round 1, static refresh, and sparring. The held-out set contains 443 questions from the production environment, and it has been confirmed not to overlap with the BeyondAIME, AIME 2024, and AIME 2025 test questions. We compute ROC AUC using the average segment confidence, select the checkpoint with the highest AUC, and choose the earlier step in case of a tie. At test time, each verifier evaluates the same fixed set of solver answers, and results are averaged over inference seeds 42–46.

Table E: Checkpoint reselection under a unified held-out rule.

BranchStepValidation mean AUCBeyondAIME mean AUCAURCRisk@20%
Round 12400.74380.78880.577248.43%
Static refresh3200.77580.78730.574746.52%
Sparring2800.77860.85640.493630.54%

After reselection, sparring's mean AUC on BeyondAIME is higher than static refresh.

The main comparison uses the better-performing static refresh, whose checkpoint selected by the unified held-out set rule is step 320.

The two static branches use the same training configuration but start from independently trained round-1 checkpoints. Static reference was selected at step 80 using the same 443-question held-out set and AUC selection rule. The sparring checkpoint in the second experiment was also selected using the same held-out set and highest-AUC rule.

Both sparring solvers were updated for 67 steps. Regeneration covered 17,398 questions and produced 173,980 responses, of which 101,585 were retained for training.

Q2. Continuation advantage of sparring over static across independent round-1 checkpoints.

Reply: To address your concern about independent training variability, we conducted two independent training experiments. In each experiment, we started from an independently trained round-1 checkpoint and performed paired static and sparring continuation training. The BeyondAIME mean AUC results are summarized in Table F.

Table F: BeyondAIME mean AUC in two independent training experiments.

Independent training experimentRound-1 mean AUCStatic branchStatic mean AUCSparring mean AUC
First group (main comparison)0.7888Static refresh0.78730.8564
Second group0.7280Static reference0.77440.7877

The results show that sparring outperforms both round 1 and the corresponding static branch in both experiments.

The main comparison uses the first experiment, in which static performs better, in order to test the effect of sparring against the stronger static control. The second experiment was added during the rebuttal period with a limited number of training steps, and was started from a relatively weaker round-1 seed, so it shows the lower end of the gain.

Q3. Full reward vs. Outcome-only under matched settings.

Reply: DISCERN achieves higher AUC and lower AURC than Outcome-only at its best-AUC checkpoint, while the risk at matched coverage is essentially the same. Please see W2 of this response (Table C) for the configuration details and the comparison with DISCERN at the same training step.

In summary, we appreciate your constructive comments and suggestions. Please let us know if you have any other questions. We look forward to your feedback!

Reviewer_KQAVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

DISCERN trains an external chunk-scoring verifier using answer-level supervision and a sparring curriculum: a temporary solver learns to obtain high verifier scores, and its regenerated responses train the next verifier. On frozen-base BeyondAIME responses, sparring improves AUC from 0.802 to 0.869 and reduces selective risk by 16.5 percentage points at 20% coverage.

Critical strengths
  • Method looks simple, easy to implement
  • Results are promising
  • Same-response comparisons and matched-coverage analyses provide meaningful evidence of improved mathematical answer selection
  • Good analysis on distinguish ranking, calibration, process correctness
Critical weaknesses
  • The method itself is not sufficiently novel. This Solver training verifiers -> verifiers train solve mechanism have been well established with self-place and judge model training with policy training
  • The value of chunk decomposition is unclear. There is no matched trained scalar, answer-only, or fixed-segmentation baseline
  • Results are also not significant.
Questions for authors

Can validation splits, checkpoint-selection rules, and train–test overlap checks be specified more clearly

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer KQAV (Part 1)

Thank you for your constructive comments and suggestions!

W1. Limited novelty: solver-verifier co-training is already established.

Reply: Thank you for your comments. We would like to first clarify that DISCERN aims to train an answer verifier that can be deployed independently. During training, the solver updates according to the verifier's scores and generates new answers. We label these answers according to the correctness of the final answer, and then use them to continue training the verifier so that it learns to identify incorrect answers that previously received high scores. At deployment, only the verifier is kept, and there is no need to access the generator's weights or logits.

Regarding related work, the work most similar to our idea that we identified is PVG [1], which mainly studies the checkability of solutions and the verifier's ability to identify deceptive answers. Its Iterative Goodharting variant also updates the solver by maximizing the verifier score, and then trains the next-round verifier using final-answer labels. DISCERN, by contrast, focuses on selective abstention for already-generated answers, and trains the solver as a sparring partner in order to improve the verifier's ranking and filtering ability over answers.

Specifically, PVG gives a scalar score to the complete answer and performs supervised pairwise ranking according to final-answer labels. DISCERN divides knowledge and reasoning segments, gives confidence for each segment, and designs training rewards by combining error detection, question difficulty, and score trajectories. These segment scores are used both to filter complete answers and to expose local judgments that can be inspected; the latter is an interpretability property rather than a separately supervised capability. PVG retrains from the initial weights each round and mixes historical answers. DISCERN uses the updated solver to generate answers and continues training the existing verifier.

We will clarify these connections and differences in the related work and consider a direct comparison in future research.

W2. Value of chunk decomposition without matched baselines.

Reply: Thank you for your concern regarding chunk decomposition. Segment outputs make the verifier's local judgments inspectable. Among 1,300 segments, agreement with human labels was 83.0%. On 67 incorrect answers, exact core-error-block localization was 49.3%, rising to 80.6% when the core block and adjacent segments are counted as hits. These are interpretability diagnostics, not filtering metrics: localization is not directly supervised, and deployment abstention relies on average segment confidence and ranking rather than pinpointing a single block. Finer localization is a promising direction for future work.

To address your concern about matched baselines, we added two controls during the rebuttal period, Scalar and Outcome-only. Scalar does not distinguish knowledge blocks from capability blocks and directly annotates scores with <score>; Outcome-only uses only the judgment result for reward.

Scalar, Outcome-only, and DISCERN use the same initial weights, training data, per-step budget, and step count. For each baseline, we first select the checkpoint with the highest AUC averaged over five inference seeds among the evaluated checkpoints. We then compare it with DISCERN at the same training step. This setting is the most favorable to the baselines: each baseline is taken at its own best-AUC checkpoint, while DISCERN is simply read off at the same step rather than separately selected.

Evaluation follows the original 160-question setting, merging BeyondAIME, AIME 2024, and AIME 2025 and ranking them uniformly by score, with results averaged over five inference seeds.

Table A: Comparison between DISCERN and Scalar at Scalar's best-AUC checkpoint.

ConfigurationStepMean AUCMean AURCRisk@20%
DISCERN2400.86850.417427.69%
Scalar2400.82220.441228.43%

As shown in Table A, at Scalar's best-AUC checkpoint (step 240), DISCERN still achieves higher AUC, lower AURC, and lower Risk@20%.

W3. Significance of the results.

Reply: After reselecting checkpoints under the same rule (see Table B in Q1 below), sparring's mean AUC on BeyondAIME is 0.8564, and static refresh is 0.7873. Compared with the reselected round 1, sparring reduces Risk@20% by 17.89 percentage points, with a paired interval of [10.01, 25.49] percentage points, which does not contain 0.

AuthorsOpenReview ↗

Rebuttal to Reviewer KQAV (Part 2)

Q1. Validation splits, checkpoint-selection rules, and train-test overlap checks.

Reply: Following your suggestion, we specified a unified checkpoint selection rule before reselection and applied it to round 1, static refresh, and sparring. The held-out set contains 443 questions from the production environment, and it has been confirmed not to overlap with the BeyondAIME, AIME 2024, and AIME 2025 test questions. We compute ROC AUC using the average segment confidence, select the checkpoint with the highest AUC, and choose the earlier step in case of a tie. At test time, each verifier evaluates the same fixed set of solver answers, and results are averaged over inference seeds 42–46.

Table B: Checkpoint reselection under a unified held-out rule.

BranchStepValidation mean AUCBeyondAIME mean AUCAURCRisk@20%
Round 12400.74380.78880.577248.43%
Static refresh3200.77580.78730.574746.52%
Sparring2800.77860.85640.493630.54%

We used a script to check the four training files Round-1, Round-2, Scalar, and Solver, totaling 282,062 lines, which after deduplication correspond to 16,558 questions, with no cases of question-stem extraction failure. After exact matching and normalized matching between training question stems and test question stems, no duplicates were found.

In summary, we appreciate your constructive comments and suggestions. Please let us know if you have any other questions. We look forward to your feedback!

Reference:

[1] Kirchner, et al. Prover-Verifier Games improve legibility of LLM outputs. arXiv:2407.13692, 2024.

AuthorsOpenReview ↗

Follow up on our reply

Dear Reviewer KQAV,

Thank you again for your time and effort placed in reviewing our paper! We would be grateful to know whether any concerns remain, and we would be happy to provide further clarification during the discussion period. We very much look forward to your reply.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global response regarding paper revision

Dear Reviewers:

We would like to thank you for your constructive comments and suggestions, which have been very helpful in improving our paper. We also appreciate your recognition of the promising results and the evidence from same-response and matched-coverage evaluations.

Following your feedback, we have made substantial revisions to the narrative and organization of the paper, clarified the training and evaluation settings, and provided additional experiments and analyses in this response. The main textual revisions are marked in red in the revised manuscript (please see the updated .pdf file). The main changes and additions are summarized below.

  • Reorganized the abstract, introduction, and subsequent sections to make the relationship among the problem, method, and experiments clearer.

  • Expanded the explanation of segment confidence, changes in local verification judgments, and the use of average confidence for answer screening. We also clarified the roles of the structured rewards and the training-time solver.

  • Clarified the distinct roles of semantic-tag agreement, error-block localization, response ranking, calibration, and selective abstention in the evaluation.

  • Corrected the description of the original Outcome-only control, clarifying that its actual training settings match those of the full-reward verifier and that only the reward function differs.

  • Provided additional details in the responses on the solver update budget, regenerated and retained training-data sizes, and the methods and results of training-test overlap checks.

  • Added a checkpoint-reselection comparison using a common held-out criterion for round 1, static refresh, and sparring, and clarified the relationship between the two static experiments and their checkpoint selection.

  • Added paired continuation results from two independently trained round-1 checkpoints. Sparring outperforms both round 1 and the corresponding static branch in both experiments. The main comparison uses the experiment with the stronger static control.

  • Added matched Scalar and Outcome-only baseline comparisons. Each baseline is evaluated at its best-AUC checkpoint and compared with DISCERN at the same training step, giving the baselines the most favorable checkpoint selection.

  • Explained the connections and differences between DISCERN and PVG in research objectives, verification outputs, and training methods. A direct comparison under unified conditions is included as a direction for future research.

  • Revised the language and presentation throughout the paper to improve clarity and consistency.

Please do feel free to let us know if you have any further questions.

Thank you very much for your valuable feedback!

Best regards,

The Authors of the Paper

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.