- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DISGUAD, a clarification-selection framework for interactive text-to-SQL. Rather than directly asking an LLM to identify ambiguous expressions, the method generates contrastive SQL interpretations, compares their canonicalized components, and maps the resulting disagreements back to user-facing clarification targets. The framework combines a query-term inventory, four probe families, component-level disagreement voting, LLM-based attribution, and a heuristic prioritization mechanism with dialogue-state tracking. The downstream SQL generator remains unchanged.
Critical strengths
- Addresses an important problem: selecting clarifications that materially affect SQL semantics under limited interaction budgets.
- Presents a coherent, modular framework that maps SQL disagreement to user-facing questions while keeping the downstream generator unchanged.
- Reports consistent improvements across two backbones and both benchmark variants, with additional benefits for two downstream SQL frameworks.
Critical weaknesses
Insufficient baselines: Missing comparisons with specialized clarification methods and compute-matched alternatives make the methodological novelty difficult to assess.
Incomplete ablations: The experiments do not isolate the contribution of explicit SQL differences and voting beyond candidate metadata and additional prompting.
Limited mechanism validation: Structural disagreement does not necessarily imply genuine user ambiguity, and lexical-overlap coverage does not establish successful ambiguity resolution.
Insufficient evaluation breadth: Results remain confined to one benchmark family with simulated users, without reported uncertainty estimates or end-to-end interaction and computational costs.
Questions for authors
What are the actual interaction and inference costs? Please report clarification-turn distributions, success at fixed turn limits, token usage, and latency. Is the candidate count held constant in each probe-family ablation?
How reliable are the intermediate signals and reported gains? Please provide a semantic audit of target matching, repeated-run or paired uncertainty estimates, and an explanation of the Table 2/Figure 3 coverage discrepancy.
Does explicit SQL disagreement outperform simpler alternatives under matched budgets? In particular, how does DISGUAD compare with direct SQL-impact ranking from the same query-term inventory, and with ordinary SQL sampling followed by the same mapper?