acceptodds
Withdrawn from ICLR 2027

ClarifyAct: Learning to Clarify When Needed and Act When Ready

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models must not only know how to solve tasks, but also determine whether a request provides sufficient information to proceed. Missing premises, unclear intent boundaries, or erroneous constraints can cause failures on otherwise solvable tasks, whereas indiscriminate clarification creates unnecessary interaction costs. We define the performance difference between complete and defective requests as the suppression gap. Controlled experiments across multiple models, using complete, defective, and information-restored requests, empirically demonstrate this gap and recovery after information restoration. Motivated by this recoverability, we propose View-Specific Group-Relative Optimization (V-SGRO), which learns protective decisions from self-generated rollouts under task-verifier feedback, without external demonstrations. V-SGRO distinguishes verification status from relative quality, combining certification, preference, and residual quality signals into a composite advantage for unified policy updates. A fixed-candidate analysis establishes conditions for joint directional progress across these objectives. Experiments show that the full configuration improves the accuracy of decisions between protection and execution while reducing premature execution. Ablations and behavioral attribution analyses further reveal distinct effects of the learning signals on challenging requests.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Clarification, Information Sufficiency, Verifier-Guided Learning, Policy Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_WMoWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how LLMs can appropriately choose between clarification and execution based on whether a user request provides sufficient information. The authors propose V-SGRO, which uses rubric-based scores to construct three training signals from self-generated candidate responses: certification, preference, and residual quality. These signals are combined for RL updates. Experiments on three benchmarks demonstrate improved accuracy in deciding when to clarify and when to execute.

Critical strengths
  1. The paper is clearly written and effectively communicates its research motivation and proposed approach.
  2. The method is highly interpretable, making it easier to understand the sources of the performance gains.
  3. The paper addresses a critical and practically relevant problem in improving the reliability of LLM assistants.
Critical weaknesses
  1. The method evaluation lacks end-to-end task completion metrics. The comparative experiments primarily rely on author-defined behavioral metrics, emphasizing agreement with predefined behavioral labels. This is insufficient to establish improvements in actual task completion. Final task success rates and interaction costs should also be reported under a matched interaction budget to assess the practical benefits and overhead of clarification.
  2. The evaluation covers a limited range of LLMs. The proposed training method is evaluated only on Qwen3-8B. Experiments with backbones from different model families and parameter scales are needed to assess its generality and robustness.
  3. The diagnostic samples are subject to selection bias. The adapted τ²-Bench retains only tasks on which the screening model succeeds with complete and information-restored requests but fails with defective requests when clarification is disallowed. Consequently, the reported large performance gaps may be influenced by this selection procedure and may not reflect the broader task distribution.
Questions for authors

See Weaknesses

Reviewer_wuhCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies when large language models should clarify a user request instead of executing it. It defines the performance drop on complete requests and defective requests, and diagnoses it on self-built and adapted benchmarks. The solution, V-SGRO, combines three advantage functions into a single policy update and trains on the model's pre-generated responses scored by a task verifier.

Critical strengths

(1) The problem to ask or to execute is practical and important. The design of full/bad/assited help to compare and analyze the tasks.

(2) Transparently discuss the data leak and edge cases.

Critical weaknesses

(1) The readability is limited. For examples,

  • Narrative break. The paper motivates the problem with an execution-level diagnostic: SG and RG. However, the evaluation of V-SGRO is decision-level only.
  • It seems that the "source" has multiple meanings in the paper. For example, "task sources", "source identifiers", "record source group", "residual source", "preference source", and "source ablation". However, it is not clearly defined what the scope is.
  • The naming of tau^2-Bench and TAU2-SG is similar. However, the information is mentioned in Sec. 2.2 and Sec 4.1, respestively.
  • The naming of metrics is confusing. For example, main context: NeedAcc and Control Acc. in Table 2.

(2) There is no concrete comparison between this paper and related works. I think figuring out the difference between this paper and similar works (and as baselines) can help the reader to clarify the novelty and contribution. The current baselines are all general preference optimization methods. No one focuses on clarification-specific methods.

Questions for authors

(1) Can you evaluate on other existing clarification benchmarks, e.g., CLAMBer, AskBench? If not, why are they not considered?

(2) AskBench + rubric-guided RLVR (Zhao et al., 2026) address intent-deficient and false-premise queries. I think the settings are similar to the MCP/MIB and MD. Can you compare it with them (as a baseline) and analyze the differences?

(3) There is no ablation for the formulation of advantage functions and hyperparameters. For example, no sigmoid gate, no score-gap weight, no centering, and other parameters. Can you report the related experiments and discuss the sensitivity of these designs?

(4) Can you discuss and analyze the experimental results? For example, TAU2-SG (97.25) is greater than in-domain SG-Bench (87.50); what is the reason why TAU2-SG jumps largely from 68.32 to 97.25; is the conclusion generalized beyond Qwen3?

Reviewer_YfbpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a method to help models decide when to ask for clarification and when they have enough information to act. Experiments were conducted with Qwen3-8B on a new benchmark to test transfer to requests adapted from tool use, and separately evaluate on dialogue tasks. Results show improved initial decisions on requests with missing or conflicting information, particularly fewer attempts to act before the request is clear.

Critical strengths
  • The paper addresses a practical problem between acting too early and asking unnecessary questions
  • The experiment setup is relatively solid. The ablation study of removing individual training signals help explain which parts of the method improve the decisions. Recording premature action and unnecessary clarification also makes the accuracy tradeoff clear
Critical weaknesses
  • The results largely measured if the model initially chose to seek clarification or proceed with the task, it's different from if it asked a useful question and completed the task afterwards.
  • The proposed full training method, honestly, is more complicated than the evidence currently justifies, though the ablation experiments are still very much appreciated.
Questions for authors

Does the trained model complete more tasks successfully when it's forced to ask its own clarification questions and use the user's response? This could setup another useful baseline for comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.