- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies explicit confidence targets: a user specifies a target accuracy p in the prompt, and the model itself decides whether to answer or abstain, with the goal that accuracy among answered questions is at least p. Prior work (ERT) attempted this with GRPO but found the model insensitive to p. The paper attributes this failure to two causes: p enters the reward only through the magnitude of the error penalty (−p/(1−p)), yet GRPO's division by the group standard deviation cancels that magnitude in the common case where a rollout group has only two distinct reward values; and the penalty diverges as p→1. The proposed method, RLEC, (i) removes the standard-deviation normalization while keeping mean centering, (ii) multiplies all rewards at a given p by (1−p)^b to bound the penalty without changing the decision rule, and (iii) linearly anneals b from 0 to 1. Experiments on Qwen3-4B-Base and Qwen3-30B-A3B-Base across eight benchmarks compare RLEC with RLVR, TruthRL, ERT, and RLCR.
Critical strengths
The diagnosis points in the right direction. Figure 5(b), which adds the standard deviation back and reproduces the overlapping curves across different values of p, is a convincing ablation — it provides direct causal evidence for the core diagnosis rather than a mere correlation. The statistics in Figure 2 on how the proportions of the three group types evolve during training are a useful new measurement, grounding the "signal loss" argument empirically rather than purely algebraically. In addition, when discussing standard-deviation normalization, Appendix A.1 also brings in work arguing for keeping normalization (Ge et al. 2026; CAL-GRPO) rather than citing selectively, which I appreciate.
Critical weaknesses
1. The methodological contribution is limited, and a key comparison is missing. That dividing by the group standard deviation cancels the reward scale is the same derivation as Dr. GRPO (arXiv:2503.20783), and removing the standard deviation has already been used in several papers; the novelty here is only the observation that the cancelled scale happens to carry p. The claim "unlike all of them" in A.1 also does not hold, since Turtel et al. (2025), cited in the same paragraph, likewise remove normalization to preserve reward magnitude. The (1−p)^b factor and the b schedule lack theoretical grounding: Eq. 19 holds for any positive factor decreasing in p, and nothing justifies why this form, why a linear schedule, or why the endpoint is 1. Moreover, by Eq. 18, b merely scales the advantages uniformly at a fixed p, so the failure of b=1 to separate targets in Section 4.4 looks more like under-training under a fixed budget (b=0.5 separates only late in Figure 5(d)), and a longer budget is never tested. Finally, by the authors' own diagnosis, the most natural fix is to mean-center within the group while switching to batch-level standard deviation; the paper neither discusses nor compares against it — if it works, both the (1−p)^b factor and the b schedule become unnecessary.
2. The experimental evidence does not support the central claim of distinguishing different targets. None of the three metrics in Table 1 measures target responsiveness: Forced Accuracy is measured at p=0, Score measures utility, and AOC penalizes only shortfall, so a model that abstains uniformly conservatively can still achieve a near-zero AOC. Section 4.4 first states that b=1 "fails to distinguish the targets" and then that the three metrics "remain broadly consistent" across b, which shows precisely that these metrics cannot capture this property. The qualitative evidence is also insufficient: Figure 3 shows only three values of p, and on DAPO-Math the 0.1 and 0.5 curves nearly coincide around 50% while 0.9 rises sharply, which looks more like a threshold switch than a graded response; the graded trend in Figure 4(a) is an average over eight benchmarks and may mask per-benchmark step responses; and the only fine-grained single-dataset result (Figure 6) is on AIME24/25, where almost every question should be abstained, making the shape of the response hardest to see. I suggest adding a metric that directly quantifies target differentiation, together with per-benchmark curves over the full grid of p.
Questions for authors
Please see the Weaknesses section.