acceptodds
Under review as a conference paper at ICLR 2027

Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information

Abstract

Trustworthy reasoning requires models to abstain when essential information is missing. Yet reasoning models can identify missing premises in their reasoning and still produce unsupported final answers. We study this detection-to-abstention gap by distinguishing insufficient-information detection from the subsequent decision to abstain. We propose Judge-Then-Solve (JTS), a training framework that makes answerability an explicit commitment before solution generation. The model first audits the available premises, then learns to either continue solving or end its reasoning with abstention according to its judgment. Supervised warm-up and reinforcement learning train this behavior through structured rewards and conditional length shaping. Experiments on a dense and a mixture-of-experts reasoning model show that JTS improves overall abstention and Abstention@Detection (A@D), while shortening responses, relative to a baseline that also uses supervised warm-up, reinforcement learning, and length shaping. Across three Qwen training seeds, JTS achieves overall abstention and A@D. These gains come with lower answer rates and accuracy on answered questions, while detection errors continue to limit overall abstention. Our results support explicit answerability commitment as an effective way to improve how reasoning models act on recognized insufficiency, a specific requirement for honest and trustworthy reasoning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large language models, LLM Honesty, Reinforcement learning, LLM hallucination, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_yWjKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the detection-to-abstention gap: reasoning models often recognize missing premises in their reasoning traces yet still produce unsupported answers. The authors separate detecting insufficiency from acting on it with three metrics (DR, OAR, A@D) and propose Judge-Then-Solve (JTS), which makes answerability an explicit commitment: the model first emits a structured judgment (contextual audit, integrity check, ANSWERABLE/UNANSWERABLE), then either abstains immediately or continues solving. Training combines supervised warm-up on GPT-5.2-generated trajectories with GRPO under format, consistency, and task rewards, plus conditional length shaping. On DeepSeek-R1-Distill-Qwen-14B and Qwen3-30B-A3B-Thinking, JTS raises A@D to 99.4–99.7% and improves overall abstention over a matched SFT+RL+length baseline while shortening responses. The gains come with lower answer rates and accuracy on well-defined questions, and detection errors remain the bottleneck — notably on MedIQ, where detection degrades even as follow-through improves.

Critical strengths

Strengths

  1. The problem decomposition is clear and the diagnostic has standalone value. The paper operationalizes the phenomenon in which a reasoning model recognizes a missing premise in its trace yet still produces an unsupported answer as the detection-to-abstention gap, and separates "failure to detect" from "detected but not abstained" via DR / OAR / A@D. The diagnostics in Figures 1–2, covering frontier models and plain-RL interventions, show that the two failure modes have distinct sources and require distinct interventions. This decomposition is directly useful for follow-up work regardless of whether one adopts the proposed method, and it is the paper's key perspective beyond generic abstention-rate improvements.

  2. The method is coherent and the key comparison is fair. JTS turns answerability into an explicit upfront commitment (Contextual Audit + Integrity Check + binary conclusion, Section 4.1), and the format / consistency / task reward terms correspond to three distinct failure modes — how the judgment is expressed, whether it is followed, and whether it is correct (Section 4.3). Conditional length shaping is grouped by the example's true label rather than the model's own judgment, preferring shorter failed abstentions and longer failed solutions (Section 4.4), with well-explained motivation that prevents wrong judgments from harvesting length bonuses. In particular, the primary baseline Plain-SFT-RL + Length includes the same supervised warm-up, RL, and length shaping, differing from JTS only in the judgment protocol, which makes the comparison clean.

  3. The experimental reporting is thorough and honest. Evaluation covers both a dense model (DeepSeek-R1-Distill-Qwen-14B) and an MoE model (Qwen3-30B-A3B-Thinking), with means and standard deviations over three training seeds. The automated detection and abstention judges are validated against five independent annotators (Fleiss' kappa > 0.85; agreement 95.67% / 96.33%). Ablations cover length shaping and the consistency reward (Table 2). The authors also candidly report the costs — answer rate drops of 4.5–4.8 points and accuracy drops of 1.3–2.6 points on well-defined questions — and the negative transfer result on MedIQ where DR decreases (Table 3), without overstating the scope of their conclusions.

Critical weaknesses
  1. Limited transferability and generalization evidence. Training uses the predominantly mathematical MIP dataset, and the judgment template is designed around numerical missing-premise problems. The cross-domain evidence suggests that what transfers is following the judgment rather than making it: on MedIQ, Qwen's A@D rises from 37.7% to 97.9% while DR falls from 40.8% to 35.0% (Table 3), leaving 65% of under-specified medical questions undetected — even though detection is the bottleneck the authors themselves identify (Section 5.2). Only contextual insufficiency is covered; other insufficiency types in AbstentionBench are neither trained on nor evaluated. Please report per-domain DR/OAR, add training or evaluation on non-mathematical insufficiency, and analyze why detection degrades on MedIQ.

  2. Insufficient comparison with training-based baselines. The only external baseline is the inference-time intervention of Liu et al. (2025), evaluated on DeepSeek only; all training-time baselines are the authors' own constructions. Song et al. (2025) — cited in the paper — is the closest training-based work: it likewise uses RL training on unanswerable math (the SUM dataset) and reports restored refusal with out-of-domain generalization. The authors' Plain RL baseline is conceptually close to their recipe, yet the correspondence is never stated, and the method is neither reproduced nor evaluated on SUM. The incremental benefit of JTS over published training approaches is therefore unclear. Please reproduce the Song et al. baseline under the same backbones and evaluation protocol, or at least evaluate JTS on the SUM test set aligned with their reported numbers.

  3. Understated and under-analyzed costs on well-defined questions. Accuracy is reported conditionally on answered questions, so erroneously abstained well-defined questions leave the denominator. By Ans. × Acc., overall well-defined correctness drops by about 6.9 points on DeepSeek and 5.5 points on Qwen relative to Plain-SFT-RL + Length — far more than the "1.3–2.6 accuracy points" stated in Section 5.2, and below the untrained Base model on Qwen (≈95.3% → 88.0%). The supporting analysis is also thin: there is no per-domain breakdown of these costs, no false-UNANSWERABLE rate or failure cases on answerable inputs, and the difficulty-binned pass@8 analysis in Appendix F.1 covers only plain RL — the authors state it "does not establish that Full JTS preserves hard-math performance." Please report overall (non-conditional) correctness with per-domain breakdowns, provide the judge's false-UNANSWERABLE rate and failure cases, and extend the difficulty analysis to Full JTS.

Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

Dear Reviewer yWjK, We sincerely thank you for your thoughtful and constructive review, your positive assessment of our detection-to-abstention decomposition, and your recognition of the methodological coherence and fairness of our main comparison. We particularly appreciate your observation that distinguishing failures to detect insufficient information from failures to act on that detection has standalone diagnostic value beyond generic abstention improvements. We agree that cross-domain evidence, comparisons to prior training-based work, and answerable-task costs deserve careful examination. Below, we provide new per-domain analyses, clarify the scope of existing comparisons, and quantify the practical trade-offs without equating improved post-detection consistency with uniformly improved overall reliability.


W1: Limited Transferability and Generalization Evidence

A1: We agree that answerability detection and post-detection abstention can generalize differently, particularly on Qwen-MedIQ. To characterize this distinction more precisely, we provide a per-domain breakdown of DR and OAR. The results reveal a more nuanced pattern: compared with Plain-SFT-RL + Length, JTS improves DR in four of five domains for each backbone, with the exceptions of DeepSeek on MIP (96.9% to 95.1%) and Qwen on MedIQ (40.8% to 35.0%). At the aggregate level (submitted Table 1), DR increases from 68.0% to 88.8% on DeepSeek and from 78.1% to 81.3% on Qwen. These findings demonstrate improvements in detection across several evaluated domains, while recognizing that detection remains a limitation on Qwen-MedIQ.

(1) Per-domain detection and abstention. The following breakdown covers MIP, MMLU-Math, MMLU-History, GPQA-Diamond, and MedIQ. Each cell reports DR / OAR (%). Plain refers to Plain-SFT-RL + Length. Results are three-training-seed means, rounded to one decimal place; the Qwen-MedIQ entries are those reported in revised Table 4.

Model / MethodMIPMMLU-MathMMLU-HistoryGPQAMedIQ
DeepSeek Plain96.9 / 95.993.2 / 86.556.4 / 23.170.0 / 62.510.2 / 2.9
DeepSeek JTS95.1 / 94.696.2 / 96.292.3 / 92.397.5 / 97.570.9 / 70.7
Qwen Plain93.0 / 90.592.5 / 92.587.2 / 75.287.5 / 85.040.8 / 15.4
Qwen JTS98.6 / 98.699.0 / 99.095.7 / 94.096.7 / 96.735.0 / 34.1

JTS achieves higher OAR than the matched Plain baseline on all three non-mathematical domains for both backbones. On DeepSeek, MedIQ OAR improves from 2.9% to 70.7% and GPQA from 62.5% to 97.5%. On Qwen, MedIQ OAR improves from 15.4% to 34.1%. Together with the DR improvements in the aggregate and in four of five domains per backbone, these results show that JTS improves both detection and post-detection abstention across most evaluated settings, while MedIQ remains a domain-specific exception for Qwen.

(2) Interpreting the Qwen-MedIQ result. Qwen-MedIQ is a domain-specific exception: DR decreases from 40.8% to 35.0%, while OAR increases from 15.4% to 34.1% and the estimated A@D rises from 37.7% to 97.9% (revised Table 4; Appendix D). A plausible explanation is the distribution mismatch between predominantly mathematical missing-premise training and clinical questions, where missing patient-specific or domain-specific context may be harder to recognize. We present this as a hypothesis, not an established causal finding. Importantly, this MedIQ result should not be generalized into a claim that JTS reduces detection overall: aggregate DR improves on both backbones, and the per-domain breakdown shows improvements in four of five domains for each backbone. The remaining MedIQ detection gap is a domain-specific challenge that warrants further study.

(3) Scope. Our transfer evaluation includes non-mathematical contextual insufficiency, but does not establish transfer to temporal uncertainty or false premises. We have included the complete DR/OAR breakdown in Section 5.5, Table 4, and stated this scope in that section and Appendix G. Broader non-mathematical training and other unanswerability types remain future work.

AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

W2: Insufficient Comparison with Training-Based Baselines

A2: We thank the reviewer for highlighting Song et al. (2025), The Hallucination Tax of Reinforcement Finetuning.

(1) Relationship to Song et al. and our choice of GRPO. Our Plain RL baseline is conceptually closely related to the task-level abstention training of Song et al. We have made this relationship explicit in Section 2 and Appendix H.1. Song et al. use PPO to train refusal on unanswerable mathematical questions; our main pipeline uses GRPO without a separate critic. We have additionally evaluated an adapted PPO implementation in our task setting, described in Section 5.2, Appendix H.1, and Table 12. This comparison addresses the training-based baseline directly while preserving the distinctions in data, batch size, and seed scope below.

(2) Reproduction of Song et al.'s PPO training approach under our matched rollout budget. We implement the PPO training approach of Song et al. on Qwen3-30B-A3B-Thinking-2507, adapting the original backbone and SUM/DeepScaleR data to our MIP training and MIP/AbstentionBench evaluation protocol. As in Song et al., PPO runs for 200 optimization steps with one rollout per question; our implementation follows their PPO training procedure while using the same MIP training data, prompts, and task rewards as our Plain RL baseline. To match the number of generated trajectories across our own baselines, we use 128 questions per PPO step (Song et al.'s original experiments report a global batch size of 1,024). The resulting rollout counts are:

  • Plain RL / JTS (GRPO): 400 steps × 8 questions × 8 responses = 25,600 trajectories.
  • PPO: 200 steps × 128 questions × 1 response = 25,600 trajectories.

We report results on MIP and AbstentionBench, aligning the Base, Plain RL, and JTS columns with the submitted ICLR version:

MethodDR ↑OAR ↑A@D ↑Avg. Len. ↓Overall Corr. ↑
Base62.754.386.71553.795.2
Plain RL (GRPO)69.563.791.61708.595.1
Song et al. PPO (adapted)69.762.089.02478.694.5
JTS (Ours)81.380.999.4404.688.0

DR, OAR, A@D, and Overall Corr. are percentages. Avg. Len. is the mean generated response length on under-specified questions. Overall Corr. is the proportion of all well-defined responses that are correct, counting unnecessary abstentions as failures. It equals Answer Rate × conditional Accuracy within one evaluation population; a product of separately averaged, rounded metrics need not recover aggregate correctness. PPO averages evaluation seeds 42 and 43 from one training run; Base uses three evaluation seeds, while Plain RL and JTS use three training seeds. Equal generated-trajectory counts do not imply equal total compute. This is a method-level PPO reproduction adapted to our backbone, data, and batch size, not a rerun of the original SUM/DeepScaleR experiments. These details and results are now in Appendix H.1, Table 12.

(3) Findings. Under a matched budget of 25,600 generated training trajectories in our evaluation setting, the reproduced PPO-based baseline reaches 62.0% OAR and 89.0% A@D, close to Plain RL (GRPO) at 63.7% OAR and 91.6% A@D. Full JTS reaches 80.9% OAR and 99.4% A@D, exceeding this PPO reproduction by 18.9 and 10.4 percentage points, respectively. Thus, the advantage of the full JTS pipeline persists against a directly implemented training-based approach grounded in Song et al., rather than only against our GRPO baselines. Since the original Song et al. experiments use different backbones and the SUM/DeepScaleR training mixture, these results establish higher abstention and A@D under our common MIP training and MIP/AbstentionBench evaluation protocol, without a claim about their original SUM benchmark. We also acknowledge JTS's lower overall correctness on answerable questions, reflecting the abstention–accuracy trade-off discussed in W3.

AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W3: Understated and Under-Analyzed Costs on Well-Defined Questions

A3: We agree that conditional accuracy excludes erroneous abstentions. We now report overall correctness on well-defined inputs, counting wrong answers and unnecessary refusals as failures:

Overall correctness = (number of correct substantive answers) / (number of all well-defined questions).

The table reports three-seed per-domain results (%), with sample-size-weighted overall averages.

DomainDeepSeek PlainDeepSeek JTSQwen PlainQwen JTS
MIP86.5185.9889.9585.71
MMLU-Math97.7490.2398.7593.23
MMLU-History93.1682.9196.5894.87
GPQA87.5062.5087.5079.17
MedIQ84.8979.1192.6785.78
Overall89.6982.8693.5187.98

(1) Full cost of erroneous abstention. Compared with Plain-SFT-RL + Length, overall well-defined correctness decreases by 6.83 percentage points on DeepSeek and 5.53 points on Qwen. These declines include unnecessary abstentions and are larger than the conditional-accuracy reductions. We have reported overall correctness alongside coverage and conditional accuracy in Section 5.2, with the full domain table and denominator definition in Appendix H.2, Table 13.

(2) Domain-specific failure modes. The costs are heterogeneous and are not solely attributable to conservative refusal. DeepSeek-GPQA has an answer-rate decrease from 97.50% to 90.00% and a conditional-accuracy decrease from 89.74% to 69.44%, yielding 87.50% to 62.50% overall correctness. On DeepSeek-MIP, the overall-correctness decrease is 0.53 points. We have discussed both reduced coverage and reduced solving accuracy in Section 5.2 and Appendix H.2.

(3) False-UNANSWERABLE judgments and failure cases. We directly checked JTS traces on ground-truth ANSWERABLE inputs for DeepSeek and Qwen, comparing explicit ANSWERABLE/UNANSWERABLE judgments with final answer/abstention labels. Across reported domains, the false-UNANSWERABLE rate (well-defined inputs judged UNANSWERABLE) exactly equals the independently measured unnecessary-abstention rate. Thus the table reports 100% − Answer Rate. Crucially, this equality was verified in both models' traces, not inferred from aggregate rates or enforced by a hard decoding rule. In these runs, unnecessary abstentions stem from erroneous upfront judgments.

The resulting false-UNANSWERABLE rates on well-defined inputs are:

DomainDeepSeek Answer RateDeepSeek False-UNANSWERABLEQwen Answer RateQwen False-UNANSWERABLE
MIP93.396.6193.656.35
MMLU-Math93.736.2794.495.51
MMLU-History93.166.8494.875.13
GPQA90.0010.0093.336.67
MedIQ92.447.5693.566.44
Overall (submitted Table 1)92.97.193.96.1

We also examined the unnecessary-abstention failure cases. Two recurring patterns are (i) treating information already provided, derivable, or conventionally implied as missing and (ii) confusing the need for domain knowledge with contextual insufficiency. These cases follow the initial judgment, locating the error in answerability assessment. We have highlighted the aggregate false-UNANSWERABLE rates and these two error patterns in Section 5.5, with the full trace-level judgment/abstention comparison and domain rates in Appendix H.3, Table 14. The categories are not presented as verbatim trajectory examples or a frequency-ranked taxonomy.

Separately, Appendix F.1 explicitly states that the difficulty-binned pass@8 analysis concerns Plain RL, not Full JTS. We have retained Full JTS hard-math evaluation as an open limitation in Section 6 and Appendix G, and removed the conclusion's implication that Plain-RL-only evidence establishes Full JTS performance on hard mathematics.

(4) Reliability versus utility. High A@D does not establish a superior overall risk–coverage or utility trade-off. Section 5.6 and Appendix H.4, Tables 15–16 now give an illustrative cost-sensitive analysis: correct answers receive 1, incorrect answers 0, and abstention on answerable inputs receives c = 0.1, 0.3, or 0.5. Using overall correctness directly, JTS exceeds Plain-SFT-RL + Length in expected utility above under-specified prevalences of 18.8%/16.4%/13.9% on DeepSeek and 30.5%/26.5%/22.1% on Qwen. These thresholds depend on the declared utilities and deployment mixture. We have qualified the conclusion and deployment language; matched-coverage evaluation and application-calibrated costs remain limitations.

Reviewer_UwdHOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The authors are interested in making reasoning models better at abstention. They start from the premise that reasoning models can already detect when a question is unanswerable, but do not abstain. To address this, they suggest training models explicitly for abstention. The main contribution is a warm-start SFT phase where the SFT data features a structure where the model must commit to abstention or answering before producing an answer. The reward structure, with format rewards, additionally reinforces this commitment. The authors separately introduce a length shaping component to the reward. In their experiments on Qwen3-30B-A3B-Thinking and Deepseek-R1-Distill-Qwen-14B, with SFT data coming from GPT-5.2, they find that their method outperforms baselines on abstention rate on unanswerable questions while suffering a small decreasing in answering rate and accuracy.

Critical strengths
  • While the authors don't emphasize this as a central contribution, the length shaping and its impact on abstention rates is interesting.
  • If the results are sound, then the authors do outperform the baselines they consider.
Critical weaknesses

There are various weakness to the paper:

  • The key issue I have with the paper is the experimental setup. From the paper's writing, first and foremost, it is not clear what the Plain-SFT-RL+Length baseline actually is. How is it precisely different from Full JTS? Is it the rewards? If so, then this would amount to a removal of the format rewards while the model is still trained on the same SFT initialization. If this is not the case, however, then the only other alternative I can imagine is that the SFT data is different. Are the models then initialized using the Qwen and Deepseek generated trajectories, mentioned at the start of Section 4.2? If this is the case, then the paper has a major confounder. Full JTS uses GPT-5.2 (and with access to ground-truth answers themselves) for SFT data. The superior capabilities of GPT-5.2, by itself, could explain the performance improvement on all fronts. What the paper then demonstrates is not that the structured SFT initialization helps, but that GPT-5.2 is a better teacher.
  • There is no ablation assessing the quality of the SFT initialized model itself.
  • The main contribution being structured SFT data is also not necessarily novel. There already exists prior work investigating abstention in the context of unanswerable questions and also propose warm-start SFT, such as the work of Zhai et al [1]. This particular paper does not impose explicit structure by asking models to output ANSWERABLE/UNANSWERABLE, but I'm not sure if this matters much as it ultimately induces the model to reason and produce an answer or abstain.
  • The length shaping is interesting, but it also is highly task-specific and introduces complexity to the method.
  • There are various issues with clarity in the writing. The main issue is the definition of Plain-SFT-RL+Length. However, there are other cases such as Section 3, where detection rate results are reported without any definition of how this is computed and "plain RL" is referenced without an associated definition or without any setup on training.
  • It is unclear to me what Section 5.5 contributes to the story.

My current assessment is given the lack of clarity on Plain-SFT-RL. If it is the case that this is trained on Qwen/Deepseek outputs, then there is a major confounder in the paper's results.

References:

  • [1] Zhai, H., Liang, J., & Kang, D.. Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 19674-19695).
Questions for authors

How exactly is Plain-SFT-RL+Length trained? If it is the case that the SFT model is different, then can you construct an experiment where GPT-5.2 generates SFT outputs without the explicit structure to establish that it is the structure that leads to superior performance?

7 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

Dear Reviewer UwdH, thank you for your careful review. We clarify the supervision setup at the center of your concern: both Plain-SFT-RL + Length and Full JTS use GPT-5.2-generated final SFT targets, with backbone generations serving only as references. We have made this construction explicit and added a matched three-seed Plain-SFT/JTS-SFT control that tests the supervised trajectory protocol without RL or length shaping (Section 5.3, Table 2). We address each concern below.


W1 & Q1: Definition of Plain-SFT-RL + Length and the GPT-5.2 Teacher Confound

A1: We apologize that the original description did not make the matched teacher setup sufficiently clear. Both Plain-SFT-RL + Length and Full JTS use GPT-5.2 to generate their final SFT targets; neither uses raw Qwen/DeepSeek generations as the final supervision. As stated in Section 4.2 and Appendix B.7, the Qwen/DeepSeek generations are backbone-specific reference responses used to guide style and length. GPT-5.2 generates the actual SFT target for each backbone-specific dataset, using the MIP question and answerability label, the corresponding reference response, and the correct answer for well-defined questions. The distinction is that Plain targets do not impose the JTS judgment protocol, whereas JTS targets require the explicit premise/context audit, an ANSWERABLE or UNANSWERABLE commitment, and the corresponding solve-or-abstain continuation. Both methods therefore have access to the same teacher and reference-answer information; the relevant difference is the supervised response protocol.

For completeness, we distinguish the full-pipeline comparison from the SFT-only structural control. Both Plain-SFT-RL + Length and Full JTS use supervised warm-up, GRPO, task-level supervision, and conditional length shaping. Full JTS additionally trains under the JTS output contract and applies format and judgment–response consistency rewards, including task-reward gating. Thus, Table 1 compares the complete JTS protocol with a matched non-JTS SFT–RL pipeline; it is not, by itself, an ablation of SFT formatting alone. The SFT-only comparison in Section 5.3, Table 2 removes these RL-stage differences. Inference does not impose a hard decoding stop.

ComponentPlain-SFT-RL + LengthFull JTS
SFT target generatorGPT-5.2GPT-5.2
Backbone-generated reference responsesStyle/length reference onlyStyle/length reference only
MIP questions and labelsMatchedMatched
Supervised warm-upYes, plain trajectoriesYes, JTS-structured trajectories
GRPO and task-level supervisionYesYes, with JTS validity/consistency gating
Conditional length shapingYesYes
JTS output contract and format/consistency rewardsNoYes

Under this matched-teacher setup, revised Table 1 reports the following three-training-seed means: on DeepSeek, OAR increases from 61.2% to 88.6% and A@D from 90.0% to 99.7%; on Qwen, OAR increases from 69.3% to 80.9% and A@D from 88.7% to 99.4%. Both methods use the same teacher; the separate SFT-only control addresses the supervised protocol directly. We have made the construction explicit in Section 4.2 and Appendix B.7 and defined the baseline in Section 5.1. Appendix B.7 distinguishes matched examples, epochs, sequence limits, and optimization settings from realized target-token counts, which need not be identical.

AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

Rebuttal by Authors (2/3)

W2: Quality of the SFT-Initialized Model and the Missing SFT-Only Ablation

A2: The requested SFT-only control has already been conducted. We report the matched Plain-SFT versus JTS-SFT comparison below, with no RL, format/consistency rewards, or length shaping. Both variants use the same input examples, GPT-5.2 supervision source, training budget, and optimization settings; the controlled difference is whether the target trajectory imposes the explicit JTS answerability judgment and its corresponding continuation. Each result is averaged over three training seeds.

MethodDeepSeek DR ↑DeepSeek OAR ↑DeepSeek A@D ↑Qwen DR ↑Qwen OAR ↑Qwen A@D ↑
Plain-SFT68.259.387.078.367.486.0
JTS-SFT80.379.298.787.687.599.8

JTS-SFT improves over Plain-SFT before any RL optimization: OAR increases by 19.9 percentage points and A@D by 11.7 points on DeepSeek, and OAR increases by 20.1 points and A@D by 13.8 points on Qwen. All entries are three-training-seed averages. Because this comparison removes RL, format/consistency rewards, and length shaping while matching the supervision source and training settings, it supports a contribution from the explicit judgment-and-continuation target beyond the teacher alone. We have included this completed control in Section 5.3, Table 2, with construction and training details in Section 4.2 and Appendix B.7. Qwen JTS-SFT has higher OAR than Full JTS, so we do not claim that subsequent RL monotonically improves every abstention metric. We also do not claim equal realized target-token counts or a matched-coverage advantage from this control.


W3: Novelty Relative to Abstain-R1 and Prior Warm-Start Abstention Training

A3: We appreciate the pointer to Zhai et al., Abstain-R1 (Findings of ACL 2026). The shared use of SFT followed by RL is not the novelty we claim. The distinction is the decision point and failure mode being addressed. Abstain-R1 studies calibrated abstention and post-refusal clarification through verifiable RL. JTS targets the earlier detection-to-abstention gap: a model may recognize that a necessary premise is missing in its reasoning trajectory, yet fail to act on that recognition and continue toward an unsupported answer. JTS makes this pre-solution decision explicit and binds it to the subsequent behavior.

Concretely, JTS places an explicit answerability judgment before solution generation: UNANSWERABLE commits the trajectory to stop reasoning and abstain, whereas ANSWERABLE commits it to proceed with solving. The training objective reinforces the correspondence between this early judgment and the subsequent response. This is different from treating abstention only as a final response behavior or optimizing the clarification that follows a refusal. Our DR/OAR/A@D decomposition further measures whether the model detects missing information and whether it follows through by abstaining.

This distinction is not merely a different output tag. The matched three-seed SFT-only experiment in A2 shows that the explicit judgment-and-continuation protocol improves A@D by 11.7–13.8 points over Plain-SFT, with RL and length shaping removed. We have cited Abstain-R1 in Section 2 and sharpened our claim around the explicit pre-solution stop/continue commitment and diagnostic separation between detection and acting on detection. We do not claim that SFT followed by RL is novel, that this control identifies a unique internal causal mechanism, or that it establishes superiority on Abstain-R1's clarification benchmarks.

Reference: Zhai et al., “Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL,” Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.985/

AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W4: Task Specificity and Added Complexity of Conditional Length Shaping

A4: Conditional length shaping is intentionally tailored to the asymmetric failure modes of insufficient-information reasoning; we do not present it as a general-purpose reward for all reasoning tasks. When an under-specified question is answered anyway, prolonged reasoning can extend unsupported assumptions, so the shaping term favors shorter failed non-abstentions. For well-defined questions that are answered incorrectly or erroneously refused, additional reasoning may help, so it favors longer failed attempts. Correct answers and appropriate abstentions receive no length adjustment. The term is bounded to [-0.1, 0.1], substantially smaller than the unit-scale task/structure rewards, and requires no additional model or inference-time module (Section 4.4 and Appendix B.3).

A key design choice is that the shaping is conditioned on each example's ground-truth answerability label, not the model's own judgment. Consequently, an incorrect judgment cannot select a favorable branch and harvest a length bonus. This makes the signal corrective rather than self-reinforcing. It remains auxiliary, and its contribution is separable from the JTS structure, as shown by the no-length ablation below.

JTS does not depend on length shaping for its high A@D. In revised Table 3, JTS without length shaping reaches 99.0% A@D on DeepSeek and 96.3% on Qwen; the full method reaches 99.7% and 99.4%, respectively. Length shaping raises OAR from 79.8% to 88.6% and 67.8% to 80.9%, while shortening under-specified responses (Appendix E). We have made its complementary, task-specific role explicit in Sections 4.4 and 5.4, separately from the SFT-only evidence in Table 2.


W5: Clarity of Detection Rate, “Plain RL,” and Training/Evaluation Definitions

A5: We have defined the metrics and Plain RL at first use in Section 3, expanded the baseline taxonomy in Section 5.1, and supplied the paired-outcome details in Appendix C.4.1.

(1) Detection Rate (DR). DR is the fraction of under-specified responses whose reasoning recognizes missing necessary information; OAR is the fraction whose final response abstains; A@D = P(final abstention | detection). Both evaluators label the same individual response. The response-level check of evaluation labels found n01 = 0 (no abstention-positive/detection-negative cases) in the checked records, so OAR/DR equals joint A@D for those evaluator labels. Separately, the expanded manual audit inspected 100 trajectories for each of five methods on each of two backbones, totaling 1,000, with no observed abstention without recognized insufficiency. Section 3, Section 5.1, and Appendix C.4.1–C.4.2 now distinguish the paired-label check, sampled human audit, and imperfect judge agreement. We do not interpret near-saturated A@D as proof of perfect reliability.

(2) Plain RL versus Plain-SFT-RL + Length. Plain RL starts from the original backbone without supervised warm-up and uses GRPO with the task-level abstention/correctness objective only: no JTS contract, format/consistency rewards, or length shaping. Plain-SFT-RL + Length starts from GPT-5.2-generated non-JTS SFT targets and applies task-level GRPO plus conditional length shaping. Full JTS adds the judgment-conditioned supervised and RL protocol described in A1. We have added these definitions in Sections 3 and 5.1, with reward details in Appendix B.2.

These are clarifications of the existing evaluation protocol; they do not change the reported metrics or experimental results.


W6: Role of the Former Section 5.5 (Exploratory Entropy Analysis)

A6: The entropy study is an auxiliary, exploratory descriptive analysis, now located in Appendix F.3. It asks whether a shorter abstention trajectory can coexist with higher next-token predictive entropy. The token-weighted mean entropy is higher for JTS + Length than Plain RL on AbstentionBench (0.8436 vs. 0.4210 nats) and MIP (0.4529 vs. 0.3339 nats). Since policies generate different content and response lengths, this does not isolate a mechanism or establish better epistemic calibration.

We have removed the standalone entropy subsection from the main text and retained its statistics, examples, and descriptive interpretation in Appendix F.3. The main experiments now focus on the full-pipeline comparison, matched SFT-only control, component ablations, domain transfer, and cost-sensitive utility (Sections 5.2–5.6).

Reviewer_8pepOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies cases where a reasoning model expresses recognition that essential information is missing but nevertheless gives an unsupported answer. Judge-Then-Solve (JTS) places an explicit answerability judgment before solving and trains subsequent behavior through supervised warm-up, reinforcement learning, structured rewards, and conditional length shaping. Evaluation distinguishes detection, overall abstention, and Abstention@Detection (A@D), and includes two reasoning models, three training seeds, related baselines, ablations, and human calibration of behavioral judges. JTS substantially increases reported abstention and A@D, but answer rates and conditional accuracy on answerable questions decrease. The work targets a meaningful behavioral inconsistency; the precision of the central metric and the attribution and utility of the improvement remain important open issues.

Critical strengths
  1. A concrete behavioral problem with a corresponding intervention. JTS targets an observable mismatch between recognizing missing premises and acting on that recognition. The explicit judgment and conditional continuation make the intended behavior testable. Importantly, the appendix states that inference does not impose a hard stop, so the reported abstention is not simply guaranteed by an external rule. This supports the relevance of learning the behavior itself.

  2. Substantial improvements beyond a minimally trained baseline. Against Plain-SFT-RL + Length, Table 1 reports overall abstention increasing from 61.2 to 88.6 for DeepSeek and from 69.3 to 80.9 for Qwen, with A@D increasing to 99.7 and 99.4, respectively. Three training seeds, human judge calibration, and a related inference-intervention comparison strengthen the empirical case. The paper also acknowledges important costs and detection failures rather than reporting abstention gains alone.

Critical weaknesses
  1. The central “gap closure” result needs direct measurement and a consistent reporting protocol. A@D is defined as the probability of abstention conditional on detection, but the main estimate uses OAR/DR, which requires abstention to be a subset of detection. The audit of 200 abstentions with no counterexamples is useful, yet it does not establish that relationship throughout every evaluation set. Nor do the reported judge-agreement rates alone resolve a residual gap of only a few tenths of a percentage point. Direct joint counts would be more informative. In addition, Figure 2 on p.4 and Table 1 on p.7 give different Base OAR/DR values—for Qwen, 21.1/67.0 versus 54.3/62.7—without a sufficiently explicit protocol distinction. These issues affect the quantitative strength of the central claim, although they do not show that the overall improvement is absent.

  2. The comparisons do not yet isolate a useful advantage of the proposed judgment protocol. There is no reported JTS-SFT-only result, and the match between the plain and JTS warm-up targets in teacher information, supervision, filtering, and training tokens is insufficiently clear. Consequently, the gain cannot be cleanly attributed to the proposed commitment mechanism rather than altered supervision or a more conservative operating point. This matters because, relative to Plain-SFT-RL + Length, both answer rate and conditional answer accuracy decrease: for DeepSeek, 97.7 to 92.9 and 91.8 to 89.2, respectively. A matched-supervision comparison and a coverage or utility comparison are needed to establish when the extra abstention provides a preferable tradeoff. The manuscript acknowledges parts of this limitation, but the conclusion still overstates overall reliability.

Questions for authors
  1. Clarification and re-analysis: Are detection and abstention evaluated on the same individual responses? Can the authors report the four joint outcome counts by method, domain, and seed, and compute A@D directly with uncertainty? Do conclusions hold on a common detected subset or in a paired transition analysis?

  2. Clarification: What distinguishes the Base evaluations in Figure 2 and Table 1? Please specify the datasets, prompts, checkpoints, sampling, and aggregation responsible for the different OAR/DR values, or correct the affected values and captions.

  3. Additional evidence that could change my rating: How much of the improvement is already present after JTS supervised warm-up? Are plain and JTS targets matched for teacher, questions, answer information, and training budget? A matched-target comparison would materially affect attribution to the judgment protocol.

  4. Additional evidence that could change my rating: At comparable answer coverage, or under an explicitly stated cost of incorrect answers and abstention, does JTS improve the tradeoff over the strongest relevant baseline? The answer would determine whether the contribution supports a practical reliability advantage or a narrower behavioral shift.

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/3)

Dear Reviewer 8pep, We sincerely thank you for your thoughtful and constructive evaluation. We appreciate your recognition of the detection-to-abstention gap as a meaningful behavioral problem and the value of making answerability judgments explicit before solution generation. We have carefully revisited your concerns regarding the reliability of A@D, the evaluation protocol, the contribution of the JTS structure, and the practical utility of increased abstention. We address each point below.


W1 & Q1: Reliability of A@D and Joint Detection–Abstention Outcomes

Yes. Detection and abstention are evaluated on the same individual generated response, not on independently sampled generations. For each response y_i, the detection evaluator examines its reasoning trace and the abstention evaluator examines its final answer; the two labels are paired at the response level, as formalized in Appendix C.4. Importantly, we checked the paired evaluation results underlying our reported metrics and confirmed that every response counted as an abstention (OAR) is also counted as detecting the missing information (DR): the set of abstention-positive responses is a subset of the detection-positive responses. We observed no abstention-positive/detection-negative cases in these evaluation results. This is a direct response-level verification of the required subset relationship, not merely an assumption inferred from aggregate DR and OAR. As a separate check, we expanded the original 200-response manual audit to 100 sampled trajectories for each of five methods (Base, Prompting, Plain RL, Plain-SFT-RL + Length, and Full JTS) on each of two backbones: 1,000 trajectories in total. The manual audit also found no instance of final abstention without recognition of missing necessary information.

Let D and A denote the paired detection and final-abstention labels. Because our response-level evaluation check confirmed A=1 implies D=1 for the evaluated outputs, we have:

A@D = P(A=1 | D=1) = P(A=1) / P(D=1) = OAR / DR.

Therefore, OAR/DR equals the directly defined conditional A@D for the paired evaluator labels in the checked evaluation records: every abstention-positive response is also detection-positive. We have documented this identity in Section 3, Section 5.1, and Appendix C.4.1. The expanded manual audit independently corroborates the subset relationship on its sample, while the five-annotator validation reports 95.67% detection and 96.33% abstention agreement with human labels (Fleiss' kappa > 0.85). These judge-agreement rates do not establish perfect labels or confidence bounds on near-saturated A@D.

We have added the same-response pairing protocol, the verified n01 = 0 result, and the four joint-outcome definitions in Appendix C.4.1. Appendix C.4.2 preserves the original 200-abstention audit and reports the expanded 5-method × 2-backbone × 100-trajectory audit separately. Across-training-seed standard deviations are reported in Appendices D–E. We do not supply a full method/domain/seed joint-count table, within-run confidence intervals, or a common-detected-subset paired test in this revision. The equality of the two A@D definitions preserves the reported comparison for the checked evaluator labels, while evaluator error remains a separate limitation. We do not interpret 99.4% versus 99.7% as a statistically resolved difference.

AuthorsOpenReview ↗

Rebuttal by Authors (2/3)

W2 & Q2: Clarification of Figure 2 and Table 1

Thank you for identifying the apparent discrepancy between Figure 2 and Table 1. The difference comes solely from evaluation scope and aggregation, not from changes in the model or inference protocol. The Base evaluations use the same prompts, model checkpoints, and sampling/decoding settings in both cases.

Specifically, Figure 2 reports results on the MIP test set only, whereas Table 1 reports sample-size-weighted aggregate results across the MIP test set and four AbstentionBench subsets (MMLU-History, MMLU-Math, GPQA-Diamond, and MedIQ).

Consequently, the Base DR/OAR difference is due to MIP-only versus sample-size-weighted MIP + four-subset evaluation, with no change in prompts, checkpoints, or sampling/decoding settings. We have made the scope explicit in the Figure 2 caption and Section 5.1, including the within-seed sample-size weighting formula. We also corrected Figure 1C to use the same three-seed aggregate Qwen outcomes as Table 1, eliminating its older illustrative outcome percentages.


W3 & Q3: Matched SFT Supervision and the Contribution of JTS-SFT

We thank the reviewer for raising the important question of whether the improvements originate from a stronger SFT teacher or from the proposed JTS structure.

We clarify that both Plain-SFT and JTS-SFT use GPT-5.2 to generate their final SFT targets, with matched training questions, answerability labels, supervision sources, training budgets, and optimization settings. Backbone-generated responses are used only as style and length references, rather than as the final supervision. Correct reference answers are provided for well-defined questions in both settings.

More importantly, we conducted a matched SFT-only comparison, without subsequent RL, format/consistency rewards, or length shaping. The two variants differ in whether the target trajectory incorporates the explicit JTS judgment-and-continuation structure. The results, averaged over three training seeds, are:

MethodDeepSeek DR ↑DeepSeek OAR ↑DeepSeek A@D ↑Qwen DR ↑Qwen OAR ↑Qwen A@D ↑
Plain-SFT68.259.387.078.367.486.0
JTS-SFT80.379.298.787.687.599.8

Notably, introducing the JTS structure alone improves OAR by 19.9 and 20.1 percentage points, and A@D by 11.7 and 13.8 percentage points, on DeepSeek and Qwen, respectively.

These results show that the stronger teacher alone does not explain the improvements: the judgment-and-continuation protocol already contributes at the SFT stage. We have included the matched three-seed control in Section 5.3, Table 2, with the common supervision and training settings in Section 4.2 and Appendix B.7. The control matches examples and optimization settings rather than asserting equal realized target-token counts. We do not infer an internal causal mechanism or monotonic improvement from SFT to RL; Qwen JTS-SFT has higher OAR than Full JTS.

AuthorsOpenReview ↗

Rebuttal by Authors (3/3)

W4 & Q4: Utility and the Abstention–Coverage Trade-off

We agree: improved abstention may reduce overall utility by withholding valid answers. We therefore add a two-dimensional cost-sensitive analysis.

(1) Well-defined utility (X-axis). Correct answers score 1, incorrect answers 0, and abstention on well-defined inputs scores c. Let a denote Answer Rate, k conditional Accuracy, and C Overall Correctness (C = a × k within one population). We use reported C directly in aggregates because products of separately averaged metrics may differ. Our illustrative setting uses c = 0.3, with 0.1 and 0.5 for sensitivity.

U_well(c) = C + c × (1 − a), c ∈ {0.1, 0.3, 0.5}; within one population C = a × k.

(2) Under-specified abstention score (Y-axis). Abstention scores 1; answering scores 0:

U_under = OAR.

At c = 0.3, overall correctness (Appendix H.2), coverage (Appendix E), and PPO results yield the following coordinates (%), also in Appendix H.4, Table 15:

BackboneMethodWell-defined Utility (X)Abstention Score (Y)
DeepSeekPlain-SFT-RL + Length90.3861.2
DeepSeekFull JTS85.0088.6
QwenPlain-SFT-RL + Length93.9969.3
QwenSong et al. PPO (adapted)94.9062.0
QwenFull JTS89.8080.9

At c = 0.3, adapted PPO on Qwen gives 94.90% well-defined utility / 62.0% abstention versus JTS 89.80% / 80.9%—an 18.9-point abstention gain for ~5.10-point utility cost. PPO averages two evaluation seeds from one training run; other trained methods average three training seeds.

(3) Sensitivity to abstention utility and under-specified-question prevalence. Let lambda be the fraction of under-specified questions in a deployment mixture, and use the same declared utility scale for both problem types:

U_total(c, lambda) = (1 − lambda) × U_well(c) + lambda × OAR.

With all quantities in percentages, we solve for the break-even prevalence lambda* at which JTS and a baseline have equal expected utility. JTS has higher expected utility when lambda > lambda*. The threshold varies with the utility assigned to unnecessary abstention:

Comparison (JTS vs. baseline)lambda* at c=0.1lambda* at c=0.3lambda* at c=0.5
DeepSeek vs. Plain-SFT-RL + Length18.8%16.4%13.9%
Qwen vs. Plain-SFT-RL + Length30.5%26.5%22.1%
Qwen vs. Song et al. PPO (adapted)~24.2%~21.2%~18.0%

For completeness, Plain / JTS well-defined utilities are 89.92 / 83.57, 90.38 / 85.00, 90.84 / 86.43 (DeepSeek) and 93.67 / 88.59, 93.99 / 89.80, 94.31 / 91.02 (Qwen), for c = 0.1/0.3/0.5. Qwen PPO yields ~94.63 / 94.90 / 95.17. Its c = 0.1/0.5 utilities and break-even thresholds are approximated from rounded correctness and c = 0.3 utility, not separately measured in our setting.

These thresholds make the practical trade-off explicit without claiming universal superiority. At c = 0.3, JTS exceeds Plain-SFT-RL + Length in expected utility only when the under-specified fraction exceeds 16.4% on DeepSeek or 26.5% on Qwen. They are arithmetic consequences of the declared rewards and rounded reported means, not estimates of real-world prevalence or clinically calibrated costs.

Importantly, JTS does not dominate all baselines under every cost or prevalence setting. The two-dimensional comparison and the c/lambda sensitivity analysis identify specific conditions under which the improved abstention on under-specified questions outweighs losses on well-defined questions. Under settings with few under-specified queries, or where unnecessary abstention is very costly, the plain baseline may be preferable. These illustrative rewards should be calibrated to the application before making deployment or safety-critical claims.

We have incorporated the utility definition and aggregate trade-off in Section 5.6, with the sensitivity table, prevalence derivation, and PPO comparison in Appendix H.4, Tables 15–16. Matched-coverage evaluation and application-calibrated utility remain open analyses; Section 6 and Appendix G now bound the deployment claims accordingly.


We thank the reviewer again for helping us improve the rigor and clarity of the paper. The matched SFT-only evidence, clarified evaluation scopes, and cost-sensitive analysis strengthen our understanding of where JTS provides benefits and what trade-offs it introduces. We hope these clarifications address the reviewer’s concerns and demonstrate the value of explicit answerability commitment as a mechanism for reasoning under insufficient information.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A Concise Paper & Rebuttal Summary

Dear Area Chair and Reviwers

Thank you for guiding the discussion and for encouraging the reviewers to revisit our responses. We are grateful for the care they took with the paper. Their questions helped us sharpen both the main result and the limits of what it establishes.

Paper Summary


A reasoning model can identify exactly what information is missing and still give an unsupported answer. We call this the detection-to-abstention gap. Judge-Then-Solve (JTS) trains the model to assess whether it has enough information, commit to that judgment, and then solve or abstain accordingly. Across five evaluation domains and three training seeds, the share of under-specified responses that recognized missing information yet still answered fell from 6.8% to 0.2% on DeepSeek and from 8.8% to 0.4% on Qwen, relative to Plain-SFT-RL + Length. The models learned this behavior without an inference-time rule forcing them to stop.

Review & Rebuttal Summary


Two reviewers currently recommend acceptance, and both reviewers who followed up during the discussion raised their ratings.

  • Reviewer yWjK (Rating 3, Confidence 3) wrote that “the diagnostic has standalone value” and “the key comparison is fair,” and praised the thorough reporting of both gains and costs. In response to their questions about transfer, prior work, and performance on answerable questions, we added a five-domain breakdown, an adapted PPO comparison, and an analysis of overall correctness that counts unnecessary abstentions as errors.
  • Reviewer 8pep (Rating 2 → 3, Confidence 3) valued the concrete behavioral problem and the substantial gains over the trained baseline. To address their questions about measurement and attribution, we verified detection and abstention on the same responses and added a matched SFT-only comparison. The reviewer subsequently raised their recommendation to weak acceptance and recognized the practical relevance of our research question, writing, “This direction is somewhat useful in wild domains.”
  • Reviewer UwdH (Rating 1 → 2, Confidence 3) found the effect of length shaping on abstention particularly interesting. They explicitly confirmed that their original main concern about the source of SFT supervision “is resolved.” With the same teacher, questions, and reference-answer access, JTS-SFT improved overall abstention by 19.9 percentage points on DeepSeek and 20.1 on Qwen over Plain-SFT (Table 2). Their remaining question is how much of this gain comes from auditing the premises versus committing to a verdict before solving. We agree that the suggested ablation would help distinguish those contributions. Table 2 establishes the benefit of training the two together, while Table 3 separately examines length shaping and consistency feedback.

Thank you again for your time and consideration. We hope this paper reaches people working with language models in medicine and other fields who face the question that motivated us. When a model recognizes that essential information is missing, can we trust it to say so and stop there?

Best regards,
The Authors of Paper 22782

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.