acceptodds
Under review as a conference paper at ICLR 2027

Default-Preserving Clarification: Repair, Regression, and Interaction Cost

Abstract

With the increasing deployment of language-model agents in complex task environments, proactive clarification has become a standard paradigm to resolve instruction ambiguities. However, an informative clarification can still make the resulting solution worse. When a language-model agent incorporates a reply by regenerating its output, it may repair a failed solution or mistakenly replace a correct one with an incorrect revision. We study this trade-off relative to an explicit default draft. A transition-based account expresses the value of asking as expected repair minus regression and interaction cost. It motivates Default-Preserving Clarification (DPC), a training-free method combining context- and cost-aware selection with a retain-or-revise instruction. In three paired runs on 100 ClarifyCodeBench variants, preservation reaches 49.0% pass@1 versus 47.3% for draft-aware rewriting, while reducing estimated incremental revision API cost by 26.9%. It repairs seven failed defaults and causes no observed regression among the 61 matched-answer pairs whose defaults already pass. With delivered questions and execution pathways held fixed, the cascade and a one-turn SAGE adaptation each solve 83 of 160 tasks, asking 29 and 34 questions, respectively. Ranking and cost studies on coding and translation panels characterize the selection-computation trade-off, with mixed transfer results. These findings support treating answer incorporation as part of the clarification policy, with benefits that depend on both the default and the executor.

Primary area:
foundation or frontier models, including LLMs
Keywords:
clarification seeking, ambiguity resolution, language model agents, uncertainty estimation, interactive decision making

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.80 over 5 reviews, above 27% of the others.
Reviewer_fLXfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how clarification can repair incorrect defaults or regress correct ones. It formalizes clarification utility through a repair–regression trade-off and proposes Default-Preserving Clarification (DPC), combining context- and cost-aware question selection with retain-or-revise prompting. Paired code experiments show that DPC achieves 49.0% pass@1, compared with 47.3% for draft-aware rewriting. Policy comparisons match SAGE’s task success with fewer questions, while ranking and transfer results reveal limitations in generalization.

Critical strengths
  1. The paper goes beyond question informativeness to examine whether clarification improves end-to-end task outcomes. By separating question selection from answer incorporation, it distinguishes the value of asking from the risk of incorporating a reply, while paired experiments isolate how incorporation affects repair and regression.

  2. The evaluation provides a nuanced account of clarification utility, jointly considering task success, interaction frequency, revision cost, and generalization. Its explicit discussion of source overlap, limited replication, partial cost accounting, and mixed transfer results helps clarify both the benefits and the boundaries of the proposed approach.

Critical weaknesses
  1. The incorporation gains warrant cautious interpretation. Preserve improves pass@1 from 47.3% to 49.0%, but the gain is concentrated in a small number of source problems. (Table 2.)

  2. The selector relies on heuristic scoring. Its uncalibrated model judgments and hand-designed cost terms are not direct estimates of the repair–regression utility. A clearer explanation of how the scoring rule reflects the proposed framework would strengthen the method.

  3. The policy comparison has limited scope. DPC matches SAGE-adapted on task success while asking fewer questions, but the shared-question setup does not fully evaluate each method’s native clarification process. (Table 3.)

Questions for authors

Could the unresolved disagreements in the matcher audit affect the reported repair–regression gains, given that reply matching determines whether a clarification answer is incorporated or the default is retained?

Reviewer_iS6EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether clarification always helps an LLM agent. Its main observation is that even a useful answer can cause the model to replace a correct solution with an incorrect one. The authors analyze this trade-off in terms of repairs, regressions, and interaction cost. It proposed DPC, which decides when to ask a question and instructs the model to keep its draft unless the answer requires a change.

The experiments assess question selection and answer incorporation separately, holding drafts and answers fixed when comparing revision methods. Across three runs on 100 ClarifyCodeBench variants, preservation improves pass@1 from 47.3% to 49.0% over draft-aware rewriting and reduces estimated revision cost by 26.9%. On a 160-variant selection panel, DPC and a SAGE adaptation both solve 83 tasks, while DPC asks 29 rather than 34 questions. Additional experiments examine ranking quality, computational cost, and transfer across datasets, with mixed results. The main execution findings use simulated replies and one primary model backend.

Critical strengths
  1. The paper draws attention to a practical problem that overall accuracy can hide: a useful clarification may fix some solutions while breaking others that were already correct. Separately tracking repairs and regressions gives a clearer picture of whether clarification actually helps. I find this perspective valuable because it could improve how clarification systems are evaluated, beyond the particular method proposed here.

  2. The paired experiment is a convincing part of the design. Each revision method receives the same draft and clarification answer, making it easier to attribute differences to how the answer is incorporated. Preservation improves pass@1 from 47.3% to 49.0% over draft-aware rewriting and reduces estimated revision cost by 26.9%. These results provide encouraging evidence that a simple, training-free change can improve both reliability and efficiency, although broader validation is still needed.

Critical weaknesses
  1. My main concern is whether the reported improvement generalizes. The 1.7-point gain over draft-aware rewriting amounts to five additional passes across 300 task–replicate pairs, with the advantage concentrated in only five source problems. Also, 44 of the 54 evaluation sources overlap with development, and the main execution study uses one unversioned model backend. The paired comparison is useful, but this limited evidence leaves me unconvinced that preservation offers a reliable advantage on unseen source problems and across models.

  2. I find the methodological contribution somewhat limited. The repair–regression identity follows directly from counting transitions between correct and incorrect outputs, while the main intervention is an instruction to retain the draft unless changes are needed. The selector does not explicitly estimate the regression risk that motivates the framework. Its practical advantage is also modest: DPC and the SAGE adaptation both solve 83 of 160 tasks, with DPC asking five fewer questions. Taken together, the current results do not establish a sufficiently substantial advance to support acceptance.

Questions for authors
  1. Could you break down the existing paired revision results by whether the source problem appeared during development, including the number of sources and the repair/regression counts in each group? This would help me judge whether the preservation benefit extends to unseen source problems or is mainly supported by sources encountered during development.

  2. What do the existing ablations establish about the value of the five-factor selection score beyond the context gate alone? In particular, is there a comparison where both selectors use the same delivered questions and Preserve executor? This would help clarify how much the selection mechanism contributes beyond the preservation instruction.

These are requests to clarify or further analyze the existing experiments. A larger replication across fixed model checkpoints would be useful future work, rather than a study I would expect during the rebuttal.

Reviewer_qbyYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how incorporating a clarification answer can repair an incorrect solution or introduce errors into an already-correct one. It proposes Default-Preserving Clarification (DPC), a training-free method that selectively asks questions and instructs the model to retain the default solution unless the answer requires revision. Experiments evaluate repairs, regressions, clarification selection, and computational cost, showing modest improvements in coding success and dataset-dependent selection performance.

Critical strengths

The observation that “an informative clarification can still make the resulting solution worse” provides an interesting motivation for this work. I would have found the paper more compelling if it had investigated and modeled the mechanisms underlying this effect in greater depth. The current evidence offers limited support for the claim and does not clearly establish when or why degradation occurs. Other factors likely influence how a model incorporates correct clarification into an existing solution. Although the proposed method attempts to mitigate their effects, explicitly identifying and analyzing these factors would strengthen the contribution.

Critical weaknesses
  1. It would be helpful for the authors to clarify L122–127 mathematically, at least in the appendix, and explain how it connects to claims such as “It is nonnegative before observation cost because the prior action remains available. Replacing that posterior optimization with a fixed generation rule removes the guarantee. …”. Mathematically, how can the availability of a posterior distribution worsen the model’s prediction compared with a prior?

  2. There is a problem with the mathematical formulation: if the gold answer is provided, then, according to the mathematics currently used here, it should almost always lead to a better outcome. I suggest that the authors introduce a variable that accounts for the discrepancy and define when such a variable is gold and leads to improvement, and under what conditions it leads to regression. Currently, this is not intuitive. The ambiguity dimension definition used in SOMA-SQL [2] might help motivate this, though it does not necessarily provide a solution to the stated problem. That said, this line could partially help answer it: “The judgment concerns the need for clarification, not whether the model can implement the task correctly.” Perhaps the judgment about the need to ask could be flawed, leading to redundant clarification questions about aspects that are not really ambiguous. Could this be modeled as a variable?

  3. Please explain in detail the motivation behind Equation 4. How did the authors arrive at this equation? How is the threshold set?

  4. The methods section needs to be written more clearly. The joint controller and cascade controller need to be explained with greater clarity, along with the motivation for each. Specifically, for the joint controller, I did not understand what the “separate context critic” refers to.

  5. In the experimental section, please clarify each baseline and clarification method, with clear descriptions of their inputs and outputs. For instance, the “one-turn SAGE adaptation” in L280 is introduced directly, with a line explaining why this comparison is important.

  6. Please clarify that IntentSim and Sample Entropy are baselines for ranking clarifications. The baselines are all mixed together, which will confuse readers.

  7. Regarding “These are local adaptations with a common backend, not results copied from the original papers,” could the authors clarify what they mean by this?

  8. I think there are many heuristics and LLM-dependent numbers used in the scoring functions that need independent evaluation. For instance, Equation 4 requires an evaluation of how well the LLM predicts its terms. I think the proposed method is very sensitive to hyperparameters, and I suggest evaluating it directly on other NL2SQL tasks that have the same problem of ambiguous NL questions mapping to multiple SQL queries. Try NL2SQL benchmarks such as Ambrosia and AmbiQT, which have annotated ambiguities for each NL question, directly with the proposed heuristics. Then identify the ambiguities associated with each question and determine whether they match the annotated ambiguities.

  9. From Figure 3(c), although the result is statistically significant, I do not think the gain of 1.7 (preserve vs. rewrite) strongly indicates that, given a gold clarification from the user, the LLM tends to deviate substantially from the intended path. This goes back to the mathematical formulation comment: if we could model a parameter whose state causes this degradation, experiments could focus on this effect to understand it further. That could be poor prior instruction-following behavior, for example. In its current form, it does not add much information beyond what is already known.

  10. “Always Ask” in Table 3 asks in 160 cases. This could also be due to a bad prompt template that prompts the LLM to take an “annoyingly” proactive approach by asking about anything. Conditioning on a taxonomy such as the one in [1] for NL2SQL could help mitigate the number of questions.

[1] https://arxiv.org/pdf/2508.15276 — AmbiSQL: Interactive Ambiguity Detection and Resolution for Text-to-SQL

[2] https://arxiv.org/pdf/2606.11424 — SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probing

Questions for authors

Please see above.

Reviewer_4NcnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Default-Preserving Clarification (DPC), a training-free approach that aims to improve clarification in LLM agents by balancing repair, regression, and costs of interaction. The method combines a heuristic selector to decide when to ask questions with a Retain-or-Revise mechanism that preserves existing solutions when possible. Experiments on coding and other tasks show modest improvements in accuracy and interaction efficiency, with mixed generalization results.

Critical strengths
  • The paper addresses an important limitation on clarification usefull for LLM agents. Incorporating additional information can introduce errors into previously correct solutions. The distinction between repair and regression provides a useful perspective on clarification effectiveness.
  • The evaluation distinguishes answer incorporation from clarification selection through controlled comparisons using shared defaults, questions, and replies. This design choice helps isolate the contribution of the preserve strategy and provides useful insights into the trade-off between task success and interaction cost.
Critical weaknesses

1- The Selector relies on 5 uncalibrated model-generated scores combined through a manually designed formula. The paper lacks ablation studies showing the individual contribution of each factor, sensitivity to their combination, or comparisons with simpler selection rules.

2- The theoretical formulation explicitly models repair, regression, and costs of interaction, but DPC does not estimate or optimise repair and regression probabilities at inference time. The proposed selection heuristic is therefore only loosely connected to the theoretical framework. Moreover the method proposed as training-free does not sufficiently investigate alternative mechanisms for controlling regression, such as decoding approaches (relevant to understand the model uncertainty), or vector steering (this last is relevant as we are strongly threating the behaviour of the model). This limits the methodological contribution beyond prompt-engineering.

3- Preserve improves pass@1 by only 1.7 percentage points over draft-aware rewriting. The improvement is concentrated in five source problems, and the reported post-hoc test does not reach conventional statistical significance (p=0.0625). The evidence is therefore insufficient to establish a robust advantage.

4- The main experiments rely on very few and expensive model backend, despite the method being highly dependent on executor behaviour. Evaluating multiple model families and sizes is necessary to establish whether the observed repair–regression trade-off generalises.

Questions for authors

Were the 5 factors used in the Selector defined from the beginning, or were they selected during the development process based on preliminary experiments? Could you clarify the rationale behind choosing these specific factors?

Reviewer_Q2cPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how a clarification answer can help fix a failed solution but also disrupt one that already works. It proposes Default-Preserving Clarification (DPC), a training-free approach that combines selective questioning with instructions to keep the default draft unless the reply calls for changes. Paired coding experiments show modest gains in accuracy and lower revision costs, while the benefits of question selection vary across datasets.

Critical strengths
  • This paper makes a useful point: even a good clarification can break a solution that was already correct. Tracking repairs and regressions separately makes this easy to see.
  • Using the same draft, question, and answer makes it easier to see what the revision prompt actually changes. Looking at when to ask separately also helps show where the gains come from.
Critical weaknesses
  • The gain over draft-aware rewriting is 1.7 percentage points and comes from just five source problems. The post hoc test across source problems is also inconclusive (p = 0.0625), and many evaluation problems share sources with the development set. With most of the evidence coming from one backend, it is hard to know how well the result carries over.
  • Most of the evidence comes from a single model backend. More experiments across different model families and sizes are needed to assess how well the method generalizes.
  • The selector relies on five uncalibrated model judgments, but the paper does not isolate their individual contributions or assess sensitivity to the chosen scoring formula.
Questions for authors

Please refer to the Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.