acceptodds
Under review as a conference paper at ICLR 2027

DualMind: When to Answer and When to Clarify in Text-to-SQL

Abstract

Text-to-SQL models often produce syntactically valid yet semantically incorrect queries, as likelihood-based decoding can silently collapse schema ambiguity without verifying user intent. We present DualMind, a neuro-symbolic framework that decouples probabilistic calibrated grounding from formal semantic consensus. It retains plausible interpretations via conformal calibration, refines candidates with certified external evidence, and emits SQL only when all survivors belong to a single semantic equivalence class. We prove a compositional error bound for adaptive multi-round clarification with no per-round error accumulation, derive a verifiable criterion for resolving semantic ambiguity, and develop an optimal questioning policy under a finite clarification budget. In a controlled synthetic grounding study with a 41,344-parameter causal Transformer and three seeds, allowing at most two simulated authoritative clarification answers increases output coverage to 96.5% while maintaining 6.2% selective error, demonstrating that DualMind can substantially resolve semantic ambiguity while preserving reliability among accepted queries.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Text-to-SQL, Neuro-Symbolic Reasoning, Semantic Ambiguity, Conformal Prediction, Interactive Clarification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hxKSOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DualMind maintains a calibrated set of candidate interpretations for a text-to-SQL request, then filters the set using trusted evidence and clarification answers. It emits SQL only when the surviving set is nonempty and all candidates have equivalent query semantics; otherwise it asks another question or abstains. The paper presents an error bound for adaptive clarification, a condition for distinguishing semantic classes, and a questioning policy under a finite budget. Evaluation uses a small Transformer on a synthetic task with a fixed database schema, 16 candidate interpretations and simulated truthful answers.

Critical strengths
  1. The paper is careful about the scope of its guarantees. Keeping one initial calibrated set and filtering it with evidence that preserves the true interpretation gives a clear correctness argument for adaptive clarification. The discussion of noisy evidence, additional pruning and the different error denominators helps avoid misinterpreting the result.

  2. Semantic agreement is a reasonable stopping condition. Different groundings can produce equivalent SQL, so requiring a unique interpretation can lead to unnecessary questions. The restricted SQL fragment provides a concrete example, and Appendix B proves the signature's exactness by constructing distinguishing database instances. Execution checks are clearly presented as implementation tests.

Critical weaknesses
  1. The technical advances relative to prior work remain limited. The adaptive error bound largely follows from the property that the true interpretation stays within the candidate set, combined with a union bound. The finite‑budget questioning policy builds on Bellman recursion. Prior work such as KnowNo already combines conformal uncertainty with help‑seeking mechanisms, and equivalence‑class active learning has studied question selection for discriminating different classes. While the authors acknowledge these connections, they have not fully articulated what new key problems are solved by porting these ideas into the SQL setting. The contribution would be more compelling if the paper explicitly identified novel guarantees or decision behaviours that cannot be achieved by existing combinations of prior techniques.

  2. Existing experiments do not fully resolve the above concerns about contribution. The model only predicts one out of sixteen plan tokens under a fixed schema, and external evidence together with clarification replies directly supply ground‑truth attributes. This setup is suitable for verifying whether the framework behaves as intended, yet its value for more realistic grounding problems remains unclear. The reported 96.5 % coverage figure also benefits from injected ground‑truth information. More direct comparisons should give interactive unique‑interpretation baselines and evidence‑filtering baselines access to identical information and query budgets. Across questioning‑policy comparisons, performance gaps are minimal: under a budget of two answers, optimal, edge‑based, entropy‑driven and random policies achieve 96.5 %, 96.5 %, 96.4 % and 96.4 % coverage respectively (Appendix D, Table 2).

Questions for authors
  1. Given a fixed candidate set, truth-preserving evidence and a known semantic equivalence relation, which result provides a substantial advance over existing conformal help-seeking and equivalence-class identification methods? Can you give a concrete example of what their combination lacks and what DualMind adds?
  2. Are there interactive unique-grounding or evidence-filtered baselines with the same evidence, available questions and answer budget? At comparable error rates among answered queries, does DualMind retain a clear advantage in coverage or question cost?
  3. Have any experiments used candidates that are not fully enumerated in advance, or evidence constraints obtained from natural-language input? If not, which result within the current finite setting establishes value beyond a correctly implemented controlled example?
Reviewer_mchBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DualMind, a framework for handling semantic ambiguity in Text-to-SQL through conformal prediction, certified evidence refinement, semantic equivalence checking, and budgeted clarification. Rather than emitting a query whenever a model produces a plausible interpretation, DualMind retains a calibrated set of candidate interpretations and generates SQL only when the surviving candidates belong to a single semantic equivalence class.

The paper establishes a compositional error bound under assumptions about initial candidate coverage, truth-preserving evidence, and sound semantic verification. It also characterizes when clarification questions can resolve semantic ambiguity and formulates optimal question selection under a finite budget.

Experiments on a controlled synthetic grounding task show that clarification can substantially increase output coverage while maintaining relatively low selective error. However, the evaluation is restricted to a small, enumerated interpretation space and a lightweight Transformer, limiting the evidence for practical effectiveness.

Critical strengths

The paper addresses an important problem in Text-to-SQL: syntactically valid queries can still be semantically incorrect because of ambiguous business terminology and schema grounding.

Distinguishing uncertainty about internal interpretations from uncertainty about query semantics is a useful design principle. Semantic consensus can avoid unnecessary abstention when different interpretations produce equivalent queries.

The theoretical analysis is clearly structured. The paper distinguishes statistical coverage guarantees from semantic correctness and explicitly accounts for failures in evidence and verification.

The experimental protocol is reproducible and includes useful stress tests for corrupted evidence, distribution shifts, and candidate truncation. The authors also report coverage, wrong-output rate, and selective error separately, avoiding misleading conclusions based on a single metric.

Critical weaknesses

Limited methodological novelty. The framework combines established techniques, including conformal prediction, evidence-based filtering, semantic equivalence checking, and dynamic programming. The main novelty lies in their integration, but the paper does not sufficiently demonstrate that this combination constitutes a substantial advance over existing ambiguity-aware Text-to-SQL and clarification methods.

The experimental setting is overly simplified. The evaluation uses two tables, four binary interpretation attributes, and only 16 possible interpretations. The Transformer predicts a plan token rather than generating SQL freely. Consequently, the experiments do not adequately represent realistic Text-to-SQL challenges involving large schemas, complex joins, nested queries, and implicit business semantics.

The baselines are insufficiently competitive. The evaluation lacks comparisons with strong pretrained LLM-based Text-to-SQL systems equipped with clarification, ambiguity-aware decoding, or alternative uncertainty estimation methods. It therefore remains unclear whether DualMind offers a meaningful advantage over simpler or existing approaches.

Questions for authors

Can the authors evaluate DualMind on realistic Text-to-SQL benchmarks, such as BIRD or Spider 2.0, using a strong pretrained LLM as the underlying model?

How does DualMind compare against existing ambiguity-aware Text-to-SQL methods and LLM-based clarification strategies at matched coverage or selective error?

Can the authors demonstrate that the proposed question-selection policy improves practical outcomes over simpler policies when clarification answers are noisy, incomplete, or costly?

How does the framework handle incomplete candidate spaces, incorrect business metadata, and SQL constructs outside the restricted fragment? What guarantees remain available in these settings?

Reviewer_StB4OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DualMind, which separates Text-to-SQL into two complementary components. First a Transformer assigns probabilities to possible schema interpretations. Split-conformal calibration constructs a set of plausible interpretations. Secondly, a certified external evidence filters those interpretations, and the system generates SQL only when all remaining candidates are semantically equivalent.

Critical strengths
  • The paper addresses an important problem in Text-to-SQL - distinguishing high-confidence predictions from interpretations that are semantically justified by available evidence.

  • The separation between probabilistic candidate selection and symbolic semantic certification is clearly motivated.

Critical weaknesses
  • The evaluation uses a synthetic 16-world grounding problem with two predefined database tables and a 41K-parameter Transformer. The model predicts categorical plans rather than generating SQL. Additionally, the experiments do not establish applicability to realistic Text-to-SQL tasks.

  • The main interactive results rely on simulated authoritative answers that directly reveal the correct interpretation attributes. The method is not evaluated with natural-language clarification responses or datasets with uncertainty involved.

  • The budget-optimal policy achieves almost identical performance to simple edge-based, entropy-based, and random-question strategies. At two clarification answers, the coverage difference between the optimal and random policies is only 0.1 percentage points. Is this expected ? if so the performance gain might not be statistically significant.

  • the main novel methodological contribution seems to be not clear

Questions for authors

How does the framework behave when clarification answers are incorrect, incomplete, or expressed in unrestricted natural language?

Under what conditions does budget-optimal question selection offer meaningful improvements over simpler heuristic or random clarification strategies?

What is the main novel methodological contribution beyond composing established statistical and symbolic techniques?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.