- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a semantic complexity-aware data synthesis framework for Text-to-SQL. It introduces a Depth-Width complexity measure to characterize cross-scope dependencies and within-scope structural complexity. Based on seven structural motifs, the framework generates targeted synthetic SQL examples and applies multi-stage validation to ensure data quality. Experiments on Spider and BIRD show improvements in execution accuracy, particularly on high-complexity queries.
Critical strengths
-
Interesting perspective. The paper explores Text-to-SQL complexity from a semantic perspective rather than relying solely on conventional syntactic metrics.
-
Well-structured framework. The proposed pipeline integrates complexity measurement, structural motif extraction, targeted data synthesis, and quality validation into a coherent framework.
-
Multi-stage data validation. The combination of executability, motif consistency, and redundancy checks provides a useful mechanism for filtering low-quality synthetic SQL examples.
-
Promising experimental results. The method improves execution accuracy on Spider and BIRD, with particularly notable gains on high-complexity queries.
Critical weaknesses
-
Limited distinction between semantic and syntactic complexity. Although the paper claims to model semantic complexity, the proposed Depth-Width metrics still rely heavily on syntactic SQL structures, such as subqueries, joins, and Boolean operators. It remains unclear whether they truly capture semantic complexity.
-
Insufficient validation of the complexity measure. The paper lacks systematic comparisons with existing complexity metrics, such as Spider hardness, AST depth, and SQL component counts, to demonstrate that Depth-Width better reflects actual Text-to-SQL difficulty.
-
Insufficient ablation of complexity-aware synthesis. The performance improvements may primarily result from generating more complex training examples rather than the proposed Depth-Width modeling. Comparisons with motif-only synthesis and conventional complexity-oriented data augmentation are needed to isolate its contribution.
-
Lack of Question-SQL semantic consistency verification. The multi-stage validation focuses mainly on SQL executability, motif consistency, and redundancy. However, it does not adequately verify whether the generated natural language questions accurately express the semantics of their corresponding SQL queries, potentially introducing noisy training pairs.
Questions for authors
-
How does the proposed Depth-Width measure better capture semantic complexity and predict actual Text-to-SQL difficulty compared to existing syntactic complexity metrics?
-
Can the authors provide ablation results comparing motif-only synthesis with the full Depth-Width-guided synthesis to demonstrate the independent contribution of complexity modeling?
Thank you for recognizing the perspective, integrated framework, and promising results. The completed experiments address your three main questions: D/W provides complementary difficulty information, its targeting improves High accuracy, and the generated pairs retain high fidelity.
1. Predictive value beyond conventional measures. We compared representations on the same fixed per-query model-failure outcomes using database-grouped held-out evaluation:
D/W achieves the highest AUROC and AUPRC among the individual representations. Adding it to the conventional features improves both metrics by 0.04. This demonstrates complementary predictive information within the evaluated setting and directly addresses the comparison with established difficulty measures. The original max/sum Width analysis concerns the separate choice of aggregation rule.
2. Independent contribution of D/W targeting. We completed the following comparison with 7B LoRA on Spider, fixing the generator, seed distribution, motif quotas, shared filters, accepted-data size, and training updates:
D/W improves High EX by 5.1 points over motif-only and 3.6 points over AST-depth targeting, with competitive overall EX and a small Low trade-off. This identifies the target controller's contribution under the shared pipeline. The original whole-system comparison and validation ablation provide complementary evidence about the overall method and its filters. We will add the controller results alongside those analyses.
3. Question-SQL fidelity and measured control. Section 4.2 generates each question conditioned on the SQL and schema and applies coverage checks. We supplemented this mechanism with a manual audit: 96 of 100 D/W pairs and 95 of 100 motif-only pairs were judged faithful. The audit supports high, comparable fidelity while the controller comparison measures the accuracy benefit of structural targeting. We will report question-SQL fidelity, executability, and motif validation as distinct quality dimensions.
The target audit finds 90.0% exact D/W agreement, with 10.0% target misses. We will describe the mechanism as soft targeting with measured adherence and distinguish this rate from High coverage and the full bucket/motif distributions.
4. Semantic interpretation and revisions. D/W is a semantically anchored structural coordinate: SELECT-scope dependencies define D, the largest specified within-scope burden defines W, and motifs connect these measurements to execution-relevant decisions. The new predictive and controller results establish its diagnostic and intervention value in the evaluated setting. Section 3.2's 87.3% stability on 2,847 output-equivalent pairs separately characterizes empirical rewrite stability.
We will correct Figure 1: the single-SELECT LEFT JOIN has D=1 under Appendix A, so the pair cannot illustrate D/W invariance. Sections 3-5 will connect the corrected definition, measured target adherence, controller comparison, and fidelity audit. The empirical contribution is a coordinate that adds difficulty information and guides synthesis toward higher pooled High accuracy under the evaluated pipeline and budget.