acceptodds
Under review as a conference paper at ICLR 2027

DATABASE-GROUNDED CLARIFICATION THROUGH ORDERED PROGRAM-WORLD EXPLORATION

Abstract

Interactive Text-to-SQL systems must do more than generate executable queries: they must identify underspecified user intent, determine which information can be obtained from the database environment, and decide what to ask the user under a limited interaction budget. Existing tool-augmented agents largely leave these decisions to free-form reasoning, which can lead to redundant questions, low-value exploration, and unsupported assumptions that silently alter the resulting SQL. We introduce DB-COPE, an inference-time framework that represents uncertain intent as explicit SQL decisions and instantiates grounded alternatives as complete program worlds. Structural validation checks that a claimed interpretive difference changes the SQL; unresolved, result-changing choices are clarified and the answers are carried forward in a canonical state for initial and follow-up queries. On the 600-task BIRD-INTERACT Full set, DB-COPE raises DeepSeek v4-Flash Phase 1 success from 16.17% to 23.50% and end-to-end Phase 2 success from 7.67% to 13.17%. DB-COPE likewise improves both phases by nine percentage points with GPT-5.6 Terra. On PRACTIQ, it reaches 82.48% nine-way accuracy, showing that the same representation transfers to static user-request classification. This result suggests that explicit SQL-decision representations can support program-grounded clarification by distinguishing user-request status.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
interactive Text-to-SQL, clarification planning, database agents, ambiguity resolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TxVVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposed DB-COPE, an inference-time framework for interactive Text-to-SQL, it distinguishes unresolved user intent via explicit SQL decision components, Grounded SQL Worlds and structural validation. DB-COPE clearly improves the Phase 1 and Phase 2 success rates of different models. And authors validates the transfer ability of request-state classification in PRACTIQ. I think this is a very interesting work.

Critical strengths
  1. The method design makes sense. Explicit component projection can identify shared unsupported assumptions across multiple candidate SQLs. And Grounded SQL Worlds + AST validation can connect ambiguity and SQL structure, which is better than free-form reasoning.

  2. I think the empirical validation is sufficient. The improvement is obvious in 2 backbone LLMs and also DeepSeek experiments cover statistical significance tests, ablation studies, budget sensitivity, and User Simulator robustness analysis.

Critical weaknesses
  1. Current ablation study can not clearly show the contribution of Grounded SQL Worlds, AST validation and interaction policy.
  2. Due to phase 2 does not have deterministic constraint verification, thus it's not very clear about how much improvement is from constraint reuse.
Questions for authors

Could you show more evidence in ablation study about removing AST validation under same budget controls to make the contribution of Grounded SQL Worlds more clear?

7 comments
AuthorsOpenReview ↗

Response to Reviewer TxVV (1/3)

Q1: Could you show more evidence in the ablation study about removing AST validation under the same budget controls to clarify the contributions of Grounded SQL Worlds?

A1: We thank the reviewer for this valuable suggestion. We agree that separating the contributions of Grounded SQL Worlds, AST contrast validation, and interaction mechanisms is important. We have therefore expanded the ablation study by adding two variants: w/o AST Contrast Validator and w/o Phase 2 Planning.

The ablation variants isolate different aspects of DB-COPE:

  • w/o Executable SQL Worlds: Retains explicit SQL decision states but represents alternative interpretations in natural language, removing complete SQL instantiations and their program-level comparisons.
  • w/o AST Contrast Validator: Retains complete Grounded SQL Worlds and SQL parsing, but disables component-fingerprint comparisons and explicit ASC/DESC contrast checks. This directly isolates the contribution of structural validation.
  • w/o Explicit Component Projection: Identifies clarification targets from differences between candidate worlds, without first projecting the request onto the eight-component decision inventory.
  • w/o Sparse Component-State Updates: Retains user answers in dialogue history but removes canonical state updates and the associated interaction gates.
  • w/o Phase 2 Planning: Removes the Phase 2 record of inherited requirements and follow-up changes, together with the gate requiring this record before clarification or submission.

Following DeepSeek's API update on October 17, 2026, the previously used DeepSeek-V4-Flash service was updated to DeepSeek-V4.1-Flash-1017. All newly conducted experiments in this revision, including the additional baseline experiments, use DeepSeek-V4.1-Flash-1017. We also reran Original ADK/ReAct and DB-COPE using this updated model to ensure a controlled comparison.

All ablation variants use the same initial interaction budget, action prices, and runtime limits. They are evaluated on the same 600 BIRD-INTERACT tasks with DeepSeek-V4.1-Flash-1017 and the Gemini 2.5 Flash User Simulator. Only the specified component is removed in each variant.

AuthorsOpenReview ↗

Response to Reviewer TxVV (2/3)

The results are:

Ablation VariantNormalized Reward (NR) ↑Phase 1 Success (%) ↑End-to-End Phase 2 Success (%) ↑Avg. Coins ConsumedAvg. Clarification Questions
Full DB-COPE0.240227.1716.6715.561.15
w/o Explicit Component Projection0.225225.1716.3315.531.10
w/o Executable SQL Worlds0.233526.5016.0015.221.14
w/o Sparse Component-State Updates0.236726.6716.6715.791.18
w/o AST Contrast Validator0.218224.6715.1715.371.10
w/o Phase 2 Planning0.230726.6714.6715.171.12

The expanded ablation reveals three findings:

  1. Grounded SQL Worlds and AST validation play complementary roles. Removing executable SQL Worlds decreases NR by 0.0067 and Phase 1 success by 0.67 percentage points. Removing AST contrast validation causes a larger decrease: NR drops by 0.0220, Phase 1 success by 2.50 percentage points, and end-to-end Phase 2 success by 1.50 percentage points. This demonstrates the importance of not only constructing program-grounded alternatives but also explicitly checking whether their declared contrasts are reflected in SQL structure.

  2. Explicit component projection helps identify unresolved intent. Removing it decreases NR by 0.0150 and Phase 1 success by 2.00 percentage points, supporting its role in exposing unsupported assumptions that may be shared across candidate queries.

  3. Interaction mechanisms contribute to multi-turn task completion. Removing Phase 2 Planning decreases end-to-end Phase 2 success by 2.00 percentage points, while removing sparse component-state updates has a smaller effect on aggregate success. These results are consistent with their respective roles in follow-up constraint management and persistent intent tracking.

Overall, these additional experiments distinguish the contributions of program-grounded alternative construction, structural contrast validation, and multi-turn interaction management. In particular, the dedicated AST ablation directly addresses the reviewer's concern and shows that structural validation makes a substantial contribution under the same budget controls.

We have incorporated the expanded ablation settings and analyses into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer TxVV (3/3)

Q2: Due to Phase 2 not having deterministic constraint verification, it is not clear how much improvement comes from constraint reuse.

A2: We thank the reviewer for raising this important concern. We agree that retaining previously confirmed decisions and deterministically verifying them against the final SQL are distinct operations. Although DB-COPE records reusable Phase 1 decisions and follow-up changes before acting in Phase 2, it does not implement a deterministic component-by-component SQL verification mechanism. We therefore provide two complementary analyses to examine the effectiveness of Phase 2 constraint management.

(1) Phase 2 planning ablation.

We evaluate the complete Phase 2 planning mechanism by removing the reusable-component and follow-up-change record, its associated prompts, and the gate requiring this record before a Phase 2 clarification or submission. All Phase 1 mechanisms, general repair rules, and the one-question Phase 2 limit remain unchanged.

As reported in the ablation study, removing Phase 2 planning decreases end-to-end Phase 2 success from 16.67% to 14.67%, a reduction of 2.00 percentage points, while Phase 1 success changes only slightly (27.17% to 26.67%). This suggests that dedicated follow-up planning contributes specifically to completing multi-turn tasks.

(2) Offline static diagnosis of constraint retention.

To examine whether previously confirmed requirements remain reflected in subsequent SQL programs, we conducted an offline, single-assistant-assisted static review of the saved Phase 1 and Phase 2 submissions, inspecting each applicable requirement individually.

Among the 163 Phase 1-successful tasks, we identified 115 confirmed requirements that remained applicable after the follow-up, across 84 tasks. Of these, 100 had been implemented in Phase 1, and 93 had an available Phase 2 submission for inspection. The remaining seven had no Phase 2 submission and were therefore excluded from this analysis.

The static review yielded the following results:

Diagnostic StageNumber of Requirements
Still-applicable confirmed requirements115
Implemented in Phase 1100
Implemented in Phase 1 with an available Phase 2 submission93
Clearly retained in the first Phase 2 submission92/93 (98.9%)
Clearly retained in the final Phase 2 submission92/93 (98.9%)
Uncertain in both submissions1

We also examined the subset involving SQL repair. Among the 17 previously implemented requirements in applicable repair trajectories, all 17 were retained in both the initial and final Phase 2 submissions.

These results provide direct evidence that previously confirmed requirements are generally preserved when the agent incorporates follow-up changes and repairs its SQL.

Together, the two analyses provide complementary evidence: the ablation evaluates the complete Phase 2 planning mechanism, while the offline static review examines the observable retention of previously implemented, still-applicable requirements. The review did not involve independent human adjudication and does not measure complete SQL correctness or isolate the causal contribution of constraint reuse. The current implementation does not provide deterministic constraint verification.

We have added the ablation results and the constraint-retention analysis to the revised manuscript and appendix.

Reviewer_X2raOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DB-COPE, a framework for handling ambiguous user requests in interactive Text-to-SQL systems. It breaks down user intent into explicit SQL decisions, generates alternative SQL queries to identify meaningful ambiguities, and asks users for clarification when necessary. The framework also maintains confirmed constraints across follow-up requests. Experiments on BIRD-INTERACT show improvements over the original ReAct-based agent.

Critical strengths
  1. The paper addresses an important practical problem in interactive Text-to-SQL systems, where agents need to identify ambiguous requests and decide when to ask users for clarification.

  2. The experimental results are promising, showing consistent improvements over the ReAct baseline across different models.

  3. The proposed framework provides a systematic way to resolve ambiguity and maintain user-confirmed constraints across multiple turns.

Critical weaknesses
  1. The technical novelty is limited, as the method mainly combines existing techniques such as intent decomposition, candidate SQL generation, user clarification, and state tracking.

  2. The evaluation lacks direct comparisons with existing interactive Text-to-SQL methods. Moreover, the ablation results show relatively small contributions from individual components.

  3. The framework does not verify whether the final SQL is consistent with the clarified user intent, which leaves an important part of the problem unresolved.

Questions for authors

How to you position the key contribution of this work? Is it disambiguation, or interaction SQL generation?

7 comments
AuthorsOpenReview ↗

Response to Reviewer X2ra (1/4)

Q1: What is the Core Contribution of DB-COPE, and How Does It Differ from Existing Approaches?

A1: We thank the reviewer for raising this important question. Our work focuses on interactive disambiguation for natural-language querying over databases. The central challenge we address is not how to translate an already well-specified request into SQL, but how to help users clarify underspecified querying intentions through interaction.

In practical database applications, users often express their needs in vague terms, such as "How have sales been recently?" The challenge is to determine what the user actually means, rather than simply generate SQL for one plausible interpretation. This motivates us to study proactive intent clarification, a problem that is less explored than schema linking and SQL generation in existing Text-to-SQL research.

The interaction setting is summarized as follows:

StageDescription
Initial inputThe user's initial natural-language request. Schema and other grounding information are obtained through the benchmark's charged environment tools.
Internal reasoningIn Phase 1, the agent analyzes SQL-related decisions, consults available database evidence, and constructs Grounded SQL Worlds. In Phase 2, a lightweight planning note separates inherited requirements from follow-up changes and records newly introduced ambiguities.
Agent's output at each roundA natural-language clarification question, or a decision to proceed to SQL submission. The agent does not present SQL queries to the user for inspection or revision.
User responseThe BIRD-INTERACT User Simulator answers according to the task's underlying reference intent. Phase 1 answers are incorporated through canonical component-state updates. In Phase 2, the answer supplements the interaction history and planning note to guide SQL generation, without a sparse component-state update.
Final outputThe agent generates and submits SQL for task-level functional evaluation by the benchmark.

Thus, SQL Worlds are internal reasoning and validation tools, while the actual interaction between the agent and the user simulator takes place in natural language.

Regarding technical novelty, we acknowledge that intent decomposition, candidate generation, clarification, and state tracking are established techniques individually. Our contribution lies in integrating them into a program-grounded interactive disambiguation process with three key mechanisms:

  1. Identifying unresolved intent: DB-COPE explicitly examines the key SQL decisions involved in a user's request, allowing it to identify missing requirements even when all candidate queries share the same implicit assumption.

  2. Understanding the consequences of ambiguity: DB-COPE constructs complete Grounded SQL Worlds to represent alternative interpretations and uses AST-based validation to verify that the declared differences are reflected in the corresponding SQL structures.

  3. Resolving ambiguity through interaction: DB-COPE asks targeted natural-language clarification questions. In Phase 1, users' answers are incorporated into a canonical decision state. In Phase 2, a lightweight planning note records inherited requirements and follow-up changes to guide SQL generation.

These mechanisms connect ambiguity identification, program-level contrast analysis, and user clarification, using the recorded decisions to guide subsequent SQL generation and follow-up planning.

Therefore, DB-COPE is not primarily concerned with interactive SQL generation. Its central contribution is interactive disambiguation, with SQL serving as an internal representation for reasoning about intent and as the downstream means of executing the clarified request.

AuthorsOpenReview ↗

Response to Reviewer X2ra (2/4)

Q2: The evaluation lacks direct comparisons with existing interactive Text-to-SQL methods. Moreover, the ablation results show relatively small contributions from individual components.

A2: We thank the reviewer for these constructive comments. We have added two adapted interactive clarification baselines and expanded the ablation study to better establish the contribution of individual mechanisms.

(1) Additional comparisons with existing interactive clarification methods.

We added Sphinteract-CQs and AmbiSQL to the BIRD-INTERACT evaluation, adapting their core clarification mechanisms to the benchmark's two-phase protocol, tool interface, and interaction budget.

  • Sphinteract-CQs (adapted): Retains Summarize–Review–Ask, four ambiguity types, eight predefined clarification examples, and SQL zero-shot configuration. Its correctness-checking loop uses charged SQL submissions, with at most four clarification rounds shared across both phases.
  • AmbiSQL (adapted): Retains the seven-category ambiguity detector, multiple-choice clarification, PreferenceTree updates, and SQL generation based on accumulated user evidence. Its submission-feedback repair and Phase 2 continuation are adapted to BIRD-INTERACT.

Both methods are evaluated using the same 600 BIRD-INTERACT Full tasks, system-agent model, Gemini 2.5 Flash User Simulator, initial database states, environment tools, action prices, budgets, and functional evaluator as the corresponding DB-COPE comparison.

The results are:

MethodPhase 1 Success (%)End-to-End Phase 2 Success (%)Reward*
Original ADK/ReAct20.5010.6717.55
Sphinteract-CQs (adapted)24.6715.0021.77
AmbiSQL (adapted)21.009.0017.40
DB-COPE27.1716.6724.02

DB-COPE achieves the highest overall performance across all three metrics. Compared with the stronger adapted baseline, Sphinteract-CQs, DB-COPE improves Phase 1 success by 2.50 percentage points, end-to-end Phase 2 success by 1.67 percentage points, and Reward* by 2.25 points. We note that Sphinteract-CQs retains a small advantage on Phase 2 data-management tasks (21.05% vs. 20.53%), indicating that the relative benefits vary across task types. Nevertheless, DB-COPE achieves better overall performance.

AuthorsOpenReview ↗

Response to Reviewer X2ra (3/4)

(2) Expanded ablation study.

We have expanded the original three ablations by adding two dedicated variants: w/o AST Contrast Validator and w/o Phase 2 Planning. This allows us to distinguish the contributions of constructing SQL Worlds from explicitly validating their structural contrasts, and to examine the role of dedicated follow-up planning. The variants are defined as follows:

  • w/o Explicit Component Projection: Identifies clarification targets from differences between candidate worlds, without first projecting the request onto the complete eight-component decision inventory.
  • w/o Executable SQL Worlds: Retains explicit component states but represents alternatives in natural language, removing complete SQL instantiations and their program-level comparisons.
  • w/o Sparse Component-State Updates: Retains user answers in dialogue history but removes canonical state updates and the associated update-before-action and unresolved-state submission gates.
  • w/o AST Contrast Validator: Retains complete SQL Worlds and SQL parsing but disables component-fingerprint comparisons and explicit ASC/DESC contrast checks.
  • w/o Phase 2 Planning: Removes the Phase 2 record of inherited requirements, follow-up changes, and newly introduced ambiguities, together with the gate requiring this record before clarification or submission.

All variants are evaluated on the same 600 tasks, using the same system-agent model, User Simulator, initial budget, and non-ablated settings.

The results are:

Ablation VariantNR (↑)Phase 1 Success (%) (↑)End-to-End Phase 2 Success (%) (↑)Avg. Coins Consumed (↓)Avg. Clarification Questions
Full DB-COPE0.240227.1716.6715.561.15
w/o Explicit Component Projection0.225225.1716.3315.531.10
w/o Executable SQL Worlds0.233526.5016.0015.221.14
w/o Sparse Component-State Updates0.236726.6716.6715.791.18
w/o AST Contrast Validator0.218224.6715.1715.371.10
w/o Phase 2 Planning0.230726.6714.6715.171.12

The expanded ablation reveals several complementary effects:

  • AST contrast validation has the largest impact on overall reward and Phase 1 success: Removing it decreases NR from 0.2402 to 0.2182 and Phase 1 success from 27.17% to 24.67%, highlighting the importance of verifying that semantic contrasts are reflected in SQL structure.
  • Explicit component projection improves ambiguity identification: Removing it decreases NR by 0.0150 and Phase 1 success by 2.00 percentage points, supporting the need to identify unresolved decisions even when candidate queries share the same assumptions.
  • SQL Worlds and Phase 2 planning contribute at different stages: Removing SQL Worlds decreases Phase 1 success by 0.67 percentage points, while removing Phase 2 planning decreases end-to-end Phase 2 success by 2.00 percentage points. This is consistent with their respective roles in program-grounded ambiguity analysis and follow-up constraint management.
  • Sparse state updates primarily support interaction management: Their removal has a smaller effect on aggregate success in this experiment, but increases the average number of user questions from 1.15 to 1.18.

Overall, the expanded ablation provides a more detailed picture of how the individual mechanisms contribute to DB-COPE. In particular, it distinguishes the role of explicit decision identification, program-level contrast validation, and multi-turn constraint management, rather than treating them as interchangeable components.

We have incorporated the additional baseline comparisons, ablation settings, and corresponding analyses into the revised manuscript and appendix.

Reviewer_pMjoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DB-COPE, an inference-time framework for interactive text-to-SQL aiming at clarifying and solving an ambiguous user feedback within a constrained budget. The framework does so by closely tracking SQL decisions (metric formula, threshold, filter scope, grouping, join path, aggregation, sorting, projection) and create contrastive SQL pairs for the agent to decide the clarification question sent to user, avoid wasting or missing information during interaction.

Critical strengths
  1. The 8 SQL decision components (eq 2) is well motivated and suitable for read-only query.
  2. The implementation for generating up to 2 SQL worlds (4.1) and fingerprint checking with SQLGlot (4.2) is sensible
Critical weaknesses
  1. Lacking baselines over the default ReAct baseline. The related work list some prior work on intent disambiguation in text-to-SQL such as Disambiguate First Parse Later and AmbiSQL (Saparina & Lapata, 2025; Ding et al., 2026) but doesn't explain why these methods couldn't be used as baselines.
  2. Table 4 shows that DB-COPE is weaker than the zero-shot baseline on PRACTIQ while using ~3.4x more tokens in total. While 1 contribution of the paper is the SQL decision components, Table 10 says that the recall of ambiguity detection in Filter Criteria, SELECT Column are far away from that of the zero-shot baseline. Though I understand that the zero-shot baseline has access to "detailed definitions" of the ambiguity classes (line 420), the same privilege could be added to PRACTIQ to see if there will be consistent performance gains.
Questions for authors
  1. Could the author run more baselines to support the experiments/explain why other baselines do not apply for the tested settings?
  2. Could the author clarify the 2nd weakness raised above?
  3. There were numerical inconsistencies in table 1. Given that all the first 6 baseline rows are consistent by Overall = (BI% * 410 + DM% * 190)/600. The two GPT-5.6-terra rows don't follow this calculation. There could have been a miscalculation that affects the paper's credibility. Could the authors clarify this?
9 comments
AuthorsOpenReview ↗

Response to Reviewer pMjo (1/3)

We sincerely thank the reviewer for the constructive feedback and for recognizing the value of the eight SQL decision components and the structural validation of Grounded SQL Worlds. Following the suggestions, we have added two interactive clarification baselines, conducted a controlled PRACTIQ evaluation with matched category definitions, and corrected the numerical inconsistencies in the original results. We address each concern below.

Q1: Could the authors run more baselines to support the experiments and explain why other baselines do not apply for the tested settings?

A1: Thank you for this suggestion. We agree that comparisons with existing clarification approaches are important for evaluating DB-COPE beyond the default ReAct baseline. We have added two adapted baselines, Sphinteract-CQs and AmbiSQL, to the BIRD-INTERACT evaluation while retaining their core clarification mechanisms:

  • Sphinteract-CQs (adapted): Retains the Summarize-Review–Ask procedure, four ambiguity types, eight predefined clarification examples, and SQL zero-shot configuration. Its correctness-checking loop uses charged SQL submissions, with at most four clarification rounds across both phases.
  • AmbiSQL (adapted): Retains the seven-category ambiguity detector, multiple-choice clarification, PreferenceTree updates, and SQL generation based on accumulated user evidence. Its submission-feedback repair and Phase 2 continuation are adapted to the BIRD-INTERACT protocol.

Experimental settings. All four methods are evaluated on the same 600 BIRD-INTERACT Full tasks, with identical system-agent model versions, User Simulator, database states, environment tools, action prices, interaction budgets, and functional evaluator.

Following DeepSeek's API update on October 17, 2026, the previously used DeepSeek-V4-Flash service was updated to DeepSeek-V4.1-Flash-1017. All newly conducted experiments in this revision, including the additional baseline experiments, use DeepSeek-V4.1-Flash-1017. We also reran Original ADK/ReAct and DB-COPE using this updated model to ensure a controlled comparison.

The results are:

MethodPhase 1 Success (%)End-to-End Phase 2 Success (%)Reward*
Original ADK/ReAct20.5010.6717.55
Sphinteract-CQs (adapted)24.6715.0021.77
AmbiSQL (adapted)21.009.0017.40
DB-COPE27.1716.6724.02

DB-COPE achieves the best performance across all three metrics. Compared with the strongest adapted baseline, Sphinteract-CQs, it improves Phase 1 success by 2.50 percentage points, end-to-end Phase 2 success by 1.67 percentage points, and Reward* by 2.25 points. These results suggest that DB-COPE's explicit SQL-decision representation, program-world exploration, and structured state management provide benefits beyond existing clarification strategies.

AuthorsOpenReview ↗

Response to Reviewer pMjo (2/3)

Q2: Could the authors clarify why DB-COPE underperforms the zero-shot baseline on PRACTIQ while using more tokens? Could the same category definitions be provided to DB-COPE?

A2: Thank you for this insightful suggestion. The reviewer has pointed out an important aspect of our original evaluation that we had overlooked. In the original PRACTIQ experiment, DB-COPE was not explicitly provided with the definitions of the nine intent categories, whereas the direct zero-shot classifier had access to these definitions. This potentially underestimated DB-COPE's capability in this diagnostic task. Following the reviewer's suggestion, we conducted an additional experiment, DB-COPE with Category Definitions, providing it with exactly the same nine-category definitions and answerability criteria as the direct zero-shot classifier. The original DB-COPE pipeline remains unchanged, including structured SQL-decision analysis, Grounded SQL World construction, AST-based validation, and the deterministic classification adapter.

An important distinction is that the four conditions involve two different inference paradigms:

  • Direct LLM Classification: The LLM directly predicts one of the nine intent categories from the user request. We evaluate definition-guided zero-shot and 3-shot/class settings.
  • SQL-World-Based Intent Inference: DB-COPE does not directly predict an intent category. Instead, it first performs structured SQL-decision analysis and constructs zero, one, or two Grounded SQL Worlds. A deterministic adapter then maps the resulting structured representation to the PRACTIQ taxonomy. We evaluate this approach both without and with the same category definitions provided to the direct zero-shot classifier.

All four conditions were evaluated using DeepSeek-V4.1-Flash-1017 on the same 2,836-example regenerated PRACTIQ lexicalOnly sample. All methods receive the same questions, schemas, and retrieved example values, without access to gold labels, reference SQL, subsequent dialogue turns, or database execution.

Classification performance.

Inference ParadigmConfiguration9-way Acc. (%)9-way Macro-F1 (%)3-way Acc. (%)3-way Macro-F1 (%)
Direct LLM ClassificationDefinition-guided (zero-shot)85.4080.7087.4187.06
Direct LLM ClassificationDefinition-guided (3-shot/class)87.3482.8589.5388.87
SQL-World-Based Intent InferenceDB-COPE (without definitions)82.9076.9685.2682.87
SQL-World-Based Intent InferenceDB-COPE (with definitions)87.8383.2590.1689.31

With matched category definitions, DB-COPE achieves 87.83% nine-way accuracy, improving over its condition without definitions by 4.94 percentage points. It outperforms direct LLM classification by 2.43 percentage points over zero-shot and 0.49 percentage points over 3-shot/class, achieving the best results across all four aggregate classification metrics on this sample.

This comparison supports the effectiveness of SQL-decision-based intent inference: rather than directly classifying the request, DB-COPE reasons over explicit SQL decisions and their program-level consequences before mapping its structured output to the intent taxonomy.

AuthorsOpenReview ↗

Response to Reviewer pMjo (3/3)

We further examine the two categories highlighted by the reviewer:

Per-category recall (%):

Intent CategoryDB-COPE (without definitions)DB-COPE (with definitions)Direct LLM Zero-shot
Ambiguous Filter Criteria53.1487.7995.38
Ambiguous SELECT Column63.7483.0493.57

Providing category definitions substantially improves recall on both categories, particularly Ambiguous Filter Criteria, where recall increases from 53.14% to 87.79%.

Token consumption:

Inference ParadigmConfigurationTotal Tokens (M)
Direct LLM ClassificationDefinition-guided (zero-shot)6.30
Direct LLM ClassificationDefinition-guided (3-shot/class)55.57
SQL-World-Based Intent InferenceDB-COPE (without definitions)20.41
SQL-World-Based Intent InferenceDB-COPE (with definitions)20.88

DB-COPE uses more tokens than direct zero-shot classification. However, its definition-enhanced version uses substantially fewer tokens than 3-shot/class while achieving better classification performance.

PRACTIQ provides a complementary diagnostic evaluation: it examines whether SQL-decision-based reasoning can recognize user intent in a static setting, while BIRD-INTERACT evaluates whether this representation enables more effective interactive clarification in an actual database environment.

We have updated the PRACTIQ results and discussion accordingly.

Q3: There were numerical inconsistencies in Table 1. Could the authors clarify the incorrect aggregate values in the two GPT-5.6 Terra rows?

A3: Thank you for carefully identifying this error. The reviewer is correct that the aggregate values in the two GPT-5.6 Terra rows of the original Table 1 were incorrectly calculated. We apologize for this mistake and have corrected the results.

The overall success rate is calculated using the task-count-weighted average:

Overall = (410 × BI + 190 × DM) / 600.

Reward* follows the definition in the paper:

Reward* = 0.7 × P1-SR + 0.3 × P2-SR.

The corrected results are:

MethodPhase 1 Overall (%)Phase 2 Overall (%)Reward*
GPT-5.6 Terra25.8916.2623.00
GPT-5.6 Terra + DB-COPE34.8225.0931.90
Absolute Gain+8.93+8.83+8.90

The underlying BI and DM results remain unchanged. We have corrected the aggregate values, improvement annotations, and corresponding statements throughout the manuscript.

We appreciate the reviewer for carefully checking the results and helping us improve the accuracy of the paper.

We thank the reviewer again for the helpful suggestions. The additional interactive clarification baselines, definition-controlled PRACTIQ evaluation, and corrected aggregate statistics have strengthened the experimental evidence for DB-COPE. We believe these revisions address the concerns raised and provide a more comprehensive evaluation of the proposed approach.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Final Summary of Contributions and Responses

Dear PCs, SACs, and ACs,

We thank all three reviewers for their constructive feedback. We summarize DB-COPE’s core contribution and our responses to the main concerns below.

Core contribution

DB-COPE connects explicit SQL decisions, complete SQL Worlds, and targeted natural-language clarification. Its decision inventory helps expose unresolved assumptions that candidate comparison alone can miss. Structural validation checks whether declared alternatives differ in SQL structure, while confirmed answers guide SQL generation and follow-up planning under a limited interaction budget.

The main concerns raised by the reviewers are as follows:

  1. Core contribution and distinction from existing approaches .

    Rebuttal: We clarify how decision decomposition, program-world exploration, and state management work together to support interactive intent disambiguation. SQL Worlds make the consequences of alternative interpretations inspectable, connecting unresolved decisions to targeted clarification (Sections 3–4; Appendix B).

  2. Missing comparisons with existing interactive clarification methods .

    Rebuttal: We add adapted Sphinteract-CQs and AmbiSQL under the same tasks, model, tools, and budgets. DB-COPE achieves higher overall success in both phases than these adapted baselines. Their retained mechanisms and adaptations to management tasks and two-phase interaction are documented explicitly (Sections 5.1–5.2; Appendix A.3).

  3. PRACTIQ underperformance, token cost, and unequal access to category definitions .

    Rebuttal: With matched definitions, DB-COPE achieves 87.83% nine-way accuracy and leads all four aggregate classification metrics on the regenerated sample. Its 20.88M-token consumption is below 3-shot/class but above zero-shot. PRACTIQ serves as an auxiliary diagnostic of representation transfer; DB-COPE’s primary setting is dynamic interactive Text-to-SQL, evaluated on BIRD-INTERACT. The additional structured reasoning introduces overhead relative to direct label prediction, but the diagnostic result shows that the SQL-decision/SQL-world representation captures request-status distinctions. We update the category-level results and token-use discussion accordingly (Section 5.4; Appendix C).

  4. Small component gains and the contribution of AST validation under matched budget controls .

    Rebuttal: We add dedicated AST-validation and Phase 2-planning ablations. Removing AST contrast checks reduces Phase 1 success by 2.50 percentage points; removing the complete Phase 2 planning mechanism reduces end-to-end Phase 2 success by 2.00 points. These experiments distinguish structural validation from world construction and evaluate follow-up planning as a whole (Section 5.3).

  5. Consistency of final SQL with clarified intent and evidence for constraint reuse .

    Rebuttal: We distinguish candidate-world AST checks from final benchmark functional evaluation and explicitly acknowledge the absence of deterministic constraint verification. An offline diagnosis finds 92/93 requirements retained in both first and final Phase 2 submissions. (Appendices B.5–B.6).

  6. Incorrect aggregate values in the GPT-5.6 Terra results .

    Rebuttal: We correct the aggregates using 410:190 task-count weighting and update the corresponding rewards and improvement statements. The underlying BI and DM rates remain unchanged (Section 5.2).

We sincerely appreciate the reviewers’ feedback, which helped us strengthen the experimental evidence and clarify the method’s scope and limitations. The reviewer who assigned a score of 3 has confirmed that their concerns have been addressed. We have also provided point-by-point responses and additional experimental evidence addressing the concerns of the reviewer who assigned a score of 2, although we have not yet received follow-up feedback from 1 of them.We respectfully ask the Area Chair to consider the revised manuscript and our detailed responses when assessing the contribution and supporting evidence.

Thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.