- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how models respond when their answers are challenged. It proposes Self-Disagreement, where the same model generates an answer, critiques it in a separate context, and then decides whether to revise it. Training rewards correct answers and correct assessments of the critique, while penalizing giving up a correct answer or refusing a valid correction.
The main experiments use Qwen3-8B and Gemma-2 9B, with a 327-question disagreement probe and nine public benchmarks. The authors also evaluate against external challengers. They report better accuracy after challenge and better retention of correct answers, without hurting initial-answer accuracy.
Critical strengths
- The paper looks at a problem that matters in actual use: getting an answer right initially is not enough if the model gives it up when challenged. Average accuracy after challenge improves from 50.0% to 56.3% on Qwen and from 19.9% to 37.8% on Gemma. I found the results against external challengers especially useful, since they show that the gains extend beyond responding to the model’s own critiques.
- The method uses one policy and existing answer verifiers, which makes the setup practical. I also appreciated the accuracy, revision, and verification controls, along with the checks that initial-answer accuracy does not drop. These make the overall result more convincing.
Critical weaknesses
- I am more convinced that the model gets better at holding its answer than at deciding when to change it. Correct yielding drops from 84.1% to 50.8% on Qwen and from 93.4% to 65.5% on Gemma. The paper acknowledges this and accounts for it in the net improvement. But the “skill” decomposition does not settle the issue, since the groups depend on each model’s initial answers and the challenges they produce. The examples and arguments being compared can therefore differ across models. The odds-ratio analysis also does not compare the base and trained models. I would want to see both models evaluated on identical initial answers and critiques before concluding that their ability to judge arguments has improved.
- I could not work out how the CAD-RL reward is defined. Its challenge contains generic doubt but no proposed answer, while several reward terms depend on whether the challenger’s answer is correct. What happens to those terms when there is no answer? If the supervision changes too, the comparison does not isolate the effect of substantive criticism. This leaves me unconvinced by the claim that roughly two thirds of the gain comes from the objection’s content. The verdict ablations also need clearer definitions of the rewards and output requirements.
Questions for authors
- When you say the initial answers are frozen and replayed, are the base and trained models given exactly the same initial answer and critique, or are these fixed only across challengers within each model? If you have an evaluation that holds the full input fixed across checkpoints, please report both correct holding and correct yielding.
- For CAD-RL, how do you define challenger correctness when the critique contains no answer? Is the verdict target always “incorrect”? How are the correction multiplier and stubbornness penalty handled?
- In Table 7, what changes between “Shaping only, no judge,” “Outcome only,” and “Judge on, ungated”? Please specify the reward terms, required outputs, and format gates so it is clear which comparisons isolate the verdict requirement.
2 comments
Dear Reviewer EJsb,
We wanted to follow up, as the discussion closes on November 18. With all your concerns and questions addressed in the revision, we kindly ask you to raise your rating. If any question remains, we would be happy to answer it. Thank you for your time.
Thank you for a careful and encouraging review.
W1 and Q1. Holding against deciding, on identical first answers and critiques. To address these, we added a new evaluation that holds the full input fixed across models, as you asked. Table 8 fixes the input only within each model, as its caption now states. In the new evaluation, every arm receives the base model's first answer and the same antithesis from Gemini-3.8-flash or Claude-Haiku-4.5, on the nine benchmarks, with each trained arm at its three training seeds, each scored on eight sampled answers per question (§4.1, Table 1's lower block; App. Q). Table 13 reports, for every arm, correct holding (keeping a right first answer against a wrong antithesis), correct yielding (taking a right antithesis over a wrong first answer), their mean and the AUC of switching. In all four model–challenger pairs, Self-Disagreement holds more than twice as often as any other arm, SCoRe included, while still taking most of the right antitheses; its mean is 8 to 19 points above every other arm, and its AUC is the highest. On Qwen3-8B against Gemini-3.8-flash, for example, holding rises from 30.6% to 78.0% and yielding falls from 97.0% to 73.4%, so the mean rises from 63.8 to 75.7.
So on identical inputs, the trained model is better at deciding when to change its answer, not only at keeping it.
W2 and Q2. CAD-RL's reward when the critique has no answer. To address these, we now state in App. C how each reward term is computed for CAD-RL, and we scoped the claim. The fixed passage proposes no answer, so it is graded as not correct: the verdict target is always "incorrect", the correction multiplier applies to every wrong first answer, and the stubbornness penalty never fires, since it needs a correct challenge. As you point out, the supervision then changes with the content, so §3 and App. F now say that removing the content of the disagreement, together with the supervision that depends on it, costs roughly half to two thirds of the gain, and §4.2 that the content is also what makes the verdict checkable. So these sections no longer credit that share to the content alone.
Q3. Table 7's rows, and which comparisons isolate the verdict requirement. To address this, we added a panel to Table 7 (App. E) that lists every row's reward terms, required outputs and format gate; we relabelled the rows and named the pairs that differ in one setting. All seven runs share every generation setting, the token limit included; only the synthesis template differs, with or without the verdict request.
So switching the judge on turns on three things at once: the requirement, its payment and that credit. The single-setting pairs separate them: the requirement alone is Full return, no verdict → Judge (the verdict required, not paid; App. E), and the payment alone is Judge → Self-Disagreement, three training seeds each, which lowers capitulation from 2.9% to 1.0% (Table 3) and, on the probe, raises correct holding by 6.2 points and lowers correct yielding by 7.6, their mean unchanged (App. Q). So each comparison now isolates one setting, and §4.2 now states what Table 3 shows.
How your review improved the paper: a new evaluation with identical inputs for every arm, with holding, yielding and the odds ratio across models (App. Q); Table 8's caption; CAD-RL's reward and the scoped claim (App. C; §3, §4.2, App. F); every Table 7 row defined (App. E); §4.2 restated by what Table 3 shows.
With all of your concerns and questions now addressed in the revision, we kindly ask you to raise your rating to 4, and your presentation, informativeness and soundness scores. Thank you.