- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
In this paper, the author investigates Capability Self-Assessment (CSA), a task that aims to predict whether a large language model (LLM) can correctly solve a given query before generating a response. To achieve this, the authors adopt a straightforward approach to constructing labeled training data and apply two established post-training techniques, supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR), to train the model for CSA. Through empirical comparisons, the authors conclude that RLVR achieves better self-assessment performance while more effectively preserving the model's original problem-solving capabilities.
Critical strengths
-
The paper is well-written and clearly organized. The proposed data curation strategy and training pipeline are straightforward and easy to follow.
-
The paper conducts extensive experiments across different model families, scales, and datasets, providing empirical evidence for the generalizability of its findings across the evaluated settings.
Critical weaknesses
-
The technical novelty of this work is limited. The paper does not introduce any fundamentally new training methods but instead directly applies existing techniques, such as SFT and RLVR, to the CSA task. Moreover, the task of predicting a model's own correctness or capability has already been extensively explored in prior research on uncertainty estimation and confidence prediction. The main experimental conclusions are also largely expected and provide limited new insights into the underlying problem.
-
The motivation for directly fine-tuning the original task-performing LLM to conduct CSA is not sufficiently justified. Since CSA is essentially formulated as a binary classification problem, a simpler and smaller auxiliary model could potentially perform the same task without modifying the original LLM. Such a design would completely preserve the original model's problem-solving capabilities while potentially achieving comparable CSA performance. Therefore, the paper's extensive investigation into capability retention appears to address a problem introduced by its own methodological choice. A more convincing justification or comparison with lightweight standalone predictors is needed to establish the necessity and practical advantages of the proposed approach.
-
Several important details of the data curation process remain unclear. For example, when constructing rationale-augmented training samples for SFT, how do the authors ensure that the generated rationales are logically consistent with the corresponding CSA labels? This concern is particularly relevant to the teacher-generated rationale setting, where a stronger model generates rationales conditioned on labels derived from the original model's performance. The resulting rationales may not faithfully reflect the original model's actual capabilities.
Additionally, it is unclear whether the authors apply any filtering to retain only examples for which the original model makes correct CSA predictions. If such filtering is performed, it could introduce selection bias, as the training process may primarily reinforce knowledge or assessment patterns already captured by the original model rather than improve its ability to recognize previously unknown capability limitations. Further clarification of the data filtering and quality-control procedures would help assess the validity of the training process and experimental conclusions.
Questions for authors
-
Why should capability retention be considered a central challenge of CSA rather than a consequence of the chosen implementation? If the CSA predictor and the problem-solving model were separated, what fundamental limitation would prevent this alternative from achieving the same objective?
-
What new insights do the experimental findings provide beyond the expected differences between SFT and regularized RL training?
4 comments
Response to Reviewer WpnK (2/2): rationales, data filtering
W3: Do the SFT rationales actually match the labels? They argue for their label by construction, since each one is written with the label given. We used two kinds on purpose. A stronger teacher writes better rationales, but they may not reflect what the target model can actually do. Self-written rationales are closer to the model's own view but lower in quality. Having both lets us see whether that trade-off matters, and in Table 2 neither consistently wins: each has the higher M-F1 in 6 of the 12 settings. We didn't check whether the rationales reflect the model's real reasoning, and the teacher's can't, because a different model writes them. That's a real limitation of training on rationales. It's also part of why we include label-only SFT, which has no rationale, and RLVR, where only the final decision is rewarded and the reasoning is never supervised.
W4: Is the training data filtered toward examples the model already gets right? No. No method filters the training data. Every method trains on the same full training set with the same labels. Task-mixed SFT adds solving examples on top, but nothing is removed. For the rationale variants, we write one rationale for each possible label and keep the one that matches, so no question is dropped. The one difference is RLVR's warm-up (DFW). It does pick a subset, but by a different rule: it keeps questions where the model's 16 sampled decisions disagree, not questions where it's correct. It uses that subset only for a short warm-up (42–152 steps) and then trains on everything.
In fact, that subset leans the opposite way from what you describe. On Math, 62–70% of it is labeled DELEGATE, compared with 20–34% of the full set, so it focuses on limits the model doesn't yet recognize. And SFT trained on the same subset first (DFW-matched SFT, Table 3) still falls short of RLVR: 0.69 / 0.71 vs. 0.84 / 0.85 on Math.
Thanks again. We hope this clears up the soundness concerns, and we'd be grateful if you'd consider it in your final assessment.
We appreciate the authors' detailed response. The additional clarifications regarding data construction and experimental settings have adequately addressed my concerns about the experimental setup. However, my primary concern regarding the paper's contributions remains. The current work focuses primarily on empirically comparing two training paradigms, SFT and RL, against several baselines, without providing a sufficiently insightful analytical framework or explanation of the underlying mechanisms that drive their performance differences. Developing a stronger theoretical motivation or well-grounded hypotheses, supported by comprehensive empirical analyses that investigate the fundamental differences between these training signals, would substantially strengthen the paper's contributions.
Further Clarification of Our Core Contribution and CSA Formulation
We thank the reviewer for acknowledging our clarifications. We respectfully emphasize that our primary contribution is not merely an empirical comparison between SFT and RL, but the proposed formulation of Capability Self-Assessment (CSA) as a joint learning problem that explicitly considers capability preservation (Section 3.1, Eq. 1).
Unlike conventional self-assessment approaches, which focus primarily on improving assessment accuracy, our formulation highlights a fundamental challenge: when the same model performs both self-assessment and problem-solving, training the assessment policy can alter the very capability being assessed. Consequently, improved assessment accuracy against fixed labels does not necessarily indicate better self-awareness. This motivates our formulation of CSA as optimizing assessment quality subject to preserving the model's underlying problem-solving capability.
From this perspective, comparing SFT and RLVR is not the goal itself, but rather a systematic investigation of how different training signals satisfy this joint objective. Our analyses of training dynamics (Appendix D.3), capability-preserving baselines (Table 3), and query-level capability stability (Section 4.3) provide empirical evidence supporting this formulation.
We agree that a deeper theoretical understanding of these training signals would further strengthen the work. Nevertheless, we believe identifying and formalizing the coupling between self-assessment learning and capability preservation constitutes an important contribution, beyond a conventional empirical comparison of training paradigms. We also believe this contribution is consistent with ICLR's emphasis on identifying meaningful research problems and advancing relevant knowledge through rigorous empirical investigation.
Response to Reviewer WpnK (1/2): separate predictor, novelty
Thank you for the review. Your main soundness concern is answered by a comparison that's already in the paper, which we should have made easier to find.
Q1, W2: Why train the model itself instead of a small separate predictor? It's the right question, and the paper tests it (Table 3, and Section 4.3, "Why retention matters"). Two options in Table 3 leave the model untouched:
Training the model itself with RLVR gets 0.84 / 0.85 on Math and 0.66 / 0.67 on Science.
So a separate predictor works, just less well: the better of the two is 0.11–0.14 M-F1 behind RLVR in every setting. The most accurate assessor turned out to be the model itself. So keeping its abilities intact isn't a problem we created. It's something we have to check. It's also simpler to deploy, since one model does both jobs.
W1: No new training method. That's correct, and intended. The paper is an analysis and doesn't claim a new method. Our contributions are an evaluation of self-assessment together with its effect on the model, a comparison of training-free methods, SFT and RL, and two uses of the result. We used standard SFT and GRPO on purpose, so the comparison reflects the training signal and not a new algorithm. The DFW warm-up is only an implementation detail, and we don't claim it as a contribution.
Q2: What's new beyond "regularized RL beats SFT"? The general response goes through this. In short: