- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how LLMs can appropriately choose between clarification and execution based on whether a user request provides sufficient information. The authors propose V-SGRO, which uses rubric-based scores to construct three training signals from self-generated candidate responses: certification, preference, and residual quality. These signals are combined for RL updates. Experiments on three benchmarks demonstrate improved accuracy in deciding when to clarify and when to execute.
Critical strengths
- The paper is clearly written and effectively communicates its research motivation and proposed approach.
- The method is highly interpretable, making it easier to understand the sources of the performance gains.
- The paper addresses a critical and practically relevant problem in improving the reliability of LLM assistants.
Critical weaknesses
- The method evaluation lacks end-to-end task completion metrics. The comparative experiments primarily rely on author-defined behavioral metrics, emphasizing agreement with predefined behavioral labels. This is insufficient to establish improvements in actual task completion. Final task success rates and interaction costs should also be reported under a matched interaction budget to assess the practical benefits and overhead of clarification.
- The evaluation covers a limited range of LLMs. The proposed training method is evaluated only on Qwen3-8B. Experiments with backbones from different model families and parameter scales are needed to assess its generality and robustness.
- The diagnostic samples are subject to selection bias. The adapted τ²-Bench retains only tasks on which the screening model succeeds with complete and information-restored requests but fails with defective requests when clarification is disallowed. Consequently, the reported large performance gaps may be influenced by this selection procedure and may not reflect the broader task distribution.
Questions for authors
See Weaknesses