- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposed DB-COPE, an inference-time framework for interactive Text-to-SQL, it distinguishes unresolved user intent via explicit SQL decision components, Grounded SQL Worlds and structural validation. DB-COPE clearly improves the Phase 1 and Phase 2 success rates of different models. And authors validates the transfer ability of request-state classification in PRACTIQ. I think this is a very interesting work.
Critical strengths
-
The method design makes sense. Explicit component projection can identify shared unsupported assumptions across multiple candidate SQLs. And Grounded SQL Worlds + AST validation can connect ambiguity and SQL structure, which is better than free-form reasoning.
-
I think the empirical validation is sufficient. The improvement is obvious in 2 backbone LLMs and also DeepSeek experiments cover statistical significance tests, ablation studies, budget sensitivity, and User Simulator robustness analysis.
Critical weaknesses
- Current ablation study can not clearly show the contribution of Grounded SQL Worlds, AST validation and interaction policy.
- Due to phase 2 does not have deterministic constraint verification, thus it's not very clear about how much improvement is from constraint reuse.
Questions for authors
Could you show more evidence in ablation study about removing AST validation under same budget controls to make the contribution of Grounded SQL Worlds more clear?
7 comments
Response to Reviewer TxVV (2/3)
The results are:
| Ablation Variant | Normalized Reward (NR) ↑ | Phase 1 Success (%) ↑ | End-to-End Phase 2 Success (%) ↑ | Avg. Coins Consumed | Avg. Clarification Questions |
|---|---|---|---|---|---|
| Full DB-COPE | 0.2402 | 27.17 | 16.67 | 15.56 | 1.15 |
| w/o Explicit Component Projection | 0.2252 | 25.17 | 16.33 | 15.53 | 1.10 |
| w/o Executable SQL Worlds | 0.2335 | 26.50 | 16.00 | 15.22 | 1.14 |
| w/o Sparse Component-State Updates | 0.2367 | 26.67 | 16.67 | 15.79 | 1.18 |
| w/o AST Contrast Validator | 0.2182 | 24.67 | 15.17 | 15.37 | 1.10 |
| w/o Phase 2 Planning | 0.2307 | 26.67 | 14.67 | 15.17 | 1.12 |
The expanded ablation reveals three findings:
-
Grounded SQL Worlds and AST validation play complementary roles. Removing executable SQL Worlds decreases NR by 0.0067 and Phase 1 success by 0.67 percentage points. Removing AST contrast validation causes a larger decrease: NR drops by 0.0220, Phase 1 success by 2.50 percentage points, and end-to-end Phase 2 success by 1.50 percentage points. This demonstrates the importance of not only constructing program-grounded alternatives but also explicitly checking whether their declared contrasts are reflected in SQL structure.
-
Explicit component projection helps identify unresolved intent. Removing it decreases NR by 0.0150 and Phase 1 success by 2.00 percentage points, supporting its role in exposing unsupported assumptions that may be shared across candidate queries.
-
Interaction mechanisms contribute to multi-turn task completion. Removing Phase 2 Planning decreases end-to-end Phase 2 success by 2.00 percentage points, while removing sparse component-state updates has a smaller effect on aggregate success. These results are consistent with their respective roles in follow-up constraint management and persistent intent tracking.
Overall, these additional experiments distinguish the contributions of program-grounded alternative construction, structural contrast validation, and multi-turn interaction management. In particular, the dedicated AST ablation directly addresses the reviewer's concern and shows that structural validation makes a substantial contribution under the same budget controls.
We have incorporated the expanded ablation settings and analyses into the revised manuscript.
Response to Reviewer TxVV (3/3)
Q2: Due to Phase 2 not having deterministic constraint verification, it is not clear how much improvement comes from constraint reuse.
A2: We thank the reviewer for raising this important concern. We agree that retaining previously confirmed decisions and deterministically verifying them against the final SQL are distinct operations. Although DB-COPE records reusable Phase 1 decisions and follow-up changes before acting in Phase 2, it does not implement a deterministic component-by-component SQL verification mechanism. We therefore provide two complementary analyses to examine the effectiveness of Phase 2 constraint management.
(1) Phase 2 planning ablation.
We evaluate the complete Phase 2 planning mechanism by removing the reusable-component and follow-up-change record, its associated prompts, and the gate requiring this record before a Phase 2 clarification or submission. All Phase 1 mechanisms, general repair rules, and the one-question Phase 2 limit remain unchanged.
As reported in the ablation study, removing Phase 2 planning decreases end-to-end Phase 2 success from 16.67% to 14.67%, a reduction of 2.00 percentage points, while Phase 1 success changes only slightly (27.17% to 26.67%). This suggests that dedicated follow-up planning contributes specifically to completing multi-turn tasks.
(2) Offline static diagnosis of constraint retention.
To examine whether previously confirmed requirements remain reflected in subsequent SQL programs, we conducted an offline, single-assistant-assisted static review of the saved Phase 1 and Phase 2 submissions, inspecting each applicable requirement individually.
Among the 163 Phase 1-successful tasks, we identified 115 confirmed requirements that remained applicable after the follow-up, across 84 tasks. Of these, 100 had been implemented in Phase 1, and 93 had an available Phase 2 submission for inspection. The remaining seven had no Phase 2 submission and were therefore excluded from this analysis.
The static review yielded the following results:
| Diagnostic Stage | Number of Requirements |
|---|---|
| Still-applicable confirmed requirements | 115 |
| Implemented in Phase 1 | 100 |
| Implemented in Phase 1 with an available Phase 2 submission | 93 |
| Clearly retained in the first Phase 2 submission | 92/93 (98.9%) |
| Clearly retained in the final Phase 2 submission | 92/93 (98.9%) |
| Uncertain in both submissions | 1 |
We also examined the subset involving SQL repair. Among the 17 previously implemented requirements in applicable repair trajectories, all 17 were retained in both the initial and final Phase 2 submissions.
These results provide direct evidence that previously confirmed requirements are generally preserved when the agent incorporates follow-up changes and repairs its SQL.
Together, the two analyses provide complementary evidence: the ablation evaluates the complete Phase 2 planning mechanism, while the offline static review examines the observable retention of previously implemented, still-applicable requirements. The review did not involve independent human adjudication and does not measure complete SQL correctness or isolate the causal contribution of constraint reuse. The current implementation does not provide deterministic constraint verification.
We have added the ablation results and the constraint-retention analysis to the revised manuscript and appendix.
Response to Reviewer TxVV (1/3)
Q1: Could you show more evidence in the ablation study about removing AST validation under the same budget controls to clarify the contributions of Grounded SQL Worlds?
A1: We thank the reviewer for this valuable suggestion. We agree that separating the contributions of Grounded SQL Worlds, AST contrast validation, and interaction mechanisms is important. We have therefore expanded the ablation study by adding two variants: w/o AST Contrast Validator and w/o Phase 2 Planning.
The ablation variants isolate different aspects of DB-COPE:
Following DeepSeek's API update on October 17, 2026, the previously used DeepSeek-V4-Flash service was updated to DeepSeek-V4.1-Flash-1017. All newly conducted experiments in this revision, including the additional baseline experiments, use DeepSeek-V4.1-Flash-1017. We also reran Original ADK/ReAct and DB-COPE using this updated model to ensure a controlled comparison.
All ablation variants use the same initial interaction budget, action prices, and runtime limits. They are evaluated on the same 600 BIRD-INTERACT tasks with DeepSeek-V4.1-Flash-1017 and the Gemini 2.5 Flash User Simulator. Only the specified component is removed in each variant.