acceptodds
Under review as a conference paper at ICLR 2027

From Syntactic to Semantic: Controlling Text-to-SQL Complexity via Semantically Anchored Targeted Synthesis

Abstract

Text-to-SQL failures are often driven not by surface SQL syntax, but by semantic structures that govern how queries are scoped, composed and constrained. We introduce a semantically grounded Depth-Width complexity coordinate, where Depth measures the longest chain of cross-scope dependencies and Width characterizes within-scope combinatorial semantic burden. To operationalize this coordinate for controlled analysis and synthesis, we define structural motifs as atomic, detectable, verifiable, and synthesizable patterns that anchor semantic complexity. We cast synthesis as a generate-and-verify loop with multi-stage quality validation to filter decorative complexity. On Spider-dev and BIRD-dev, our motif taxonomy captures real-world structure with high coverage, reaching 70.6% and 88.0% overall, respectively, and achieving near-complete coverage on high-complexity subsets. Finally, we propose a complexity-controlled targeted synthesis pipeline that injects motifs into SQL skeletons, instantiates parameters via data probing, and produces executable training data annotated with Depth, Width, and motif labels for expansion toward structurally complex semantic regions. Training on validated synthetic data yields consistent execution-accuracy gains, with improvements concentrated in high-complexity regions and in Depth-Width buckets corresponding to these failure-prone structures.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-SQL, Structural Reasoning, Execution Semantics, Targeted Synthesis, Depth-Width Control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZVdmOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a semantic complexity-aware data synthesis framework for Text-to-SQL. It introduces a Depth-Width complexity measure to characterize cross-scope dependencies and within-scope structural complexity. Based on seven structural motifs, the framework generates targeted synthetic SQL examples and applies multi-stage validation to ensure data quality. Experiments on Spider and BIRD show improvements in execution accuracy, particularly on high-complexity queries.

Critical strengths
  1. Interesting perspective. The paper explores Text-to-SQL complexity from a semantic perspective rather than relying solely on conventional syntactic metrics.

  2. Well-structured framework. The proposed pipeline integrates complexity measurement, structural motif extraction, targeted data synthesis, and quality validation into a coherent framework.

  3. Multi-stage data validation. The combination of executability, motif consistency, and redundancy checks provides a useful mechanism for filtering low-quality synthetic SQL examples.

  4. Promising experimental results. The method improves execution accuracy on Spider and BIRD, with particularly notable gains on high-complexity queries.

Critical weaknesses
  1. Limited distinction between semantic and syntactic complexity. Although the paper claims to model semantic complexity, the proposed Depth-Width metrics still rely heavily on syntactic SQL structures, such as subqueries, joins, and Boolean operators. It remains unclear whether they truly capture semantic complexity.

  2. Insufficient validation of the complexity measure. The paper lacks systematic comparisons with existing complexity metrics, such as Spider hardness, AST depth, and SQL component counts, to demonstrate that Depth-Width better reflects actual Text-to-SQL difficulty.

  3. Insufficient ablation of complexity-aware synthesis. The performance improvements may primarily result from generating more complex training examples rather than the proposed Depth-Width modeling. Comparisons with motif-only synthesis and conventional complexity-oriented data augmentation are needed to isolate its contribution.

  4. Lack of Question-SQL semantic consistency verification. The multi-stage validation focuses mainly on SQL executability, motif consistency, and redundancy. However, it does not adequately verify whether the generated natural language questions accurately express the semantics of their corresponding SQL queries, potentially introducing noisy training pairs.

Questions for authors
  1. How does the proposed Depth-Width measure better capture semantic complexity and predict actual Text-to-SQL difficulty compared to existing syntactic complexity metrics?

  2. Can the authors provide ablation results comparing motif-only synthesis with the full Depth-Width-guided synthesis to demonstrate the independent contribution of complexity modeling?

1 comment

Thank you for recognizing the perspective, integrated framework, and promising results. The completed experiments address your three main questions: D/W provides complementary difficulty information, its targeting improves High accuracy, and the generated pairs retain high fidelity.

1. Predictive value beyond conventional measures. We compared representations on the same fixed per-query model-failure outcomes using database-grouped held-out evaluation:

RepresentationAUROCFailure AUPRC
AST depth0.650.40
Component count0.630.38
Spider hardness0.670.42
D/W0.710.47
Conventional features together0.680.44
Conventional features + D/W0.720.48

D/W achieves the highest AUROC and AUPRC among the individual representations. Adding it to the conventional features improves both metrics by 0.04. This demonstrates complementary predictive information within the evaluated setting and directly addresses the comparison with established difficulty measures. The original max/sum Width analysis concerns the separate choice of aggregation rule.

2. Independent contribution of D/W targeting. We completed the following comparison with 7B LoRA on Spider, fixing the generator, seed distribution, motif quotas, shared filters, accepted-data size, and training updates:

SettingAll EXHigh EXLow EX
D/W + motif81.270.483.8
Motif-only80.565.384.0
AST-depth + motif81.066.884.4

D/W improves High EX by 5.1 points over motif-only and 3.6 points over AST-depth targeting, with competitive overall EX and a small Low trade-off. This identifies the target controller's contribution under the shared pipeline. The original whole-system comparison and validation ablation provide complementary evidence about the overall method and its filters. We will add the controller results alongside those analyses.

3. Question-SQL fidelity and measured control. Section 4.2 generates each question conditioned on the SQL and schema and applies coverage checks. We supplemented this mechanism with a manual audit: 96 of 100 D/W pairs and 95 of 100 motif-only pairs were judged faithful. The audit supports high, comparable fidelity while the controller comparison measures the accuracy benefit of structural targeting. We will report question-SQL fidelity, executability, and motif validation as distinct quality dimensions.

The target audit finds 90.0% exact D/W agreement, with 10.0% target misses. We will describe the mechanism as soft targeting with measured adherence and distinguish this rate from High coverage and the full bucket/motif distributions.

4. Semantic interpretation and revisions. D/W is a semantically anchored structural coordinate: SELECT-scope dependencies define D, the largest specified within-scope burden defines W, and motifs connect these measurements to execution-relevant decisions. The new predictive and controller results establish its diagnostic and intervention value in the evaluated setting. Section 3.2's 87.3% stability on 2,847 output-equivalent pairs separately characterizes empirical rewrite stability.

We will correct Figure 1: the single-SELECT LEFT JOIN has D=1 under Appendix A, so the pair cannot illustrate D/W invariance. Sections 3-5 will connect the corrected definition, measured target adherence, controller comparison, and fidelity audit. The empirical contribution is a coordinate that adds difficulty information and guides synthesis toward higher pooled High accuracy under the evaluated pipeline and budget.

Reviewer_H43mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to improve Text-to-SQL models' ability to handle complex SQL queries. The authors propose a Depth–Width complexity coordinate and seven structural motifs to guide the targeted generation of SQL training data using LLMs. The generated data is checked for executability, motif consistency, and redundancy before being mixed with the original training data for supervised fine-tuning. Experiments on Spider-dev and BIRD-dev show that the method improves execution accuracy, with larger gains on high-complexity queries.

Critical strengths
  1. The research problem is clear and practically relevant. This paper focuses on the limitations of Text-to-SQL models in handling complex queries and discusses the potential shortcomings of existing training data in covering complex structures. The authors propose a method for generating training data targeting complex SQL structures. The problem is clearly defined and directly relates to improving Text-to-SQL models' ability to handle complex queries.
  2. The proposed Depth–Width coordinate makes query complexity easier to understand. It describes complexity from two views: Depth measures cross-scope dependencies, while Width measures how multiple structures interact within the same scope. Compared with other metrics, it gives a more detailed and meaningful description of query complexity. The experiments show that it is related to model performance and can guide the generation of complex training data.
Critical weaknesses
  1. Important details of the data synthesis process are missing. The paper explains how to generate a sample after a target Depth–Width bucket and motifs are given, but not how the seed, target bucket, or motifs are chosen. Without these details, it is difficult to reproduce the method or determine whether the generated data has the intended complexity.
  2. The main experiments lack sufficient reporting on the training-data distribution. The paper does not report the distribution of the training data used for Table 4, including the number of samples in each Depth–Width bucket and the motif distribution. It also does not state whether a sample is rejected when its final Depth–Width values do not match the target. Therefore, it is difficult to assess whether the gains in each bucket are associated with increased training coverage of the corresponding complexity region and whether the targeted synthesis achieves its intended data distribution.
Questions for authors
  1. Does the final training set reflect the intended generation targets? Please report the proportion of high-complexity samples, the sample count in each Depth–Width bucket, and the distribution of the seven motifs after all validation stages, and compare them with the original training data. These statistics are needed to show whether targeted synthesis increases coverage of the intended complexity regions and whether this increase is consistent with the bucket-level gains.
  2. How do the authors validate the redundancy check? Two queries may return the same result on one database even when a motif is not redundant. For example, if all employees in the database belong to the Sales or HR department, removing a filter that asks for employees in these departments would not change the result, even though the filter is still required by the question. Do the authors test queries on multiple database instances or manually inspect cases involving NULL values, duplicate records, or particular data distributions?
  3. How are the target bucket (D*, W*) and target motif set M* sampled? Section 4.2 assumes a target bucket and motif set but does not explain how they are selected. Please specify the sampling probabilities for each target bucket and motif.
  4. How does complexity-bucket sampling differ in practice from direct random sampling? Section 4.4 appears to sample buckets according to their natural frequencies in Dmix and then sample instances within each bucket. If this is the case, the final probability of selecting each sample may be the same as that of uniform-over-instances sampling from the entire dataset.
  5. What is the practical cost of synthetic data generation? The paper reports that only 5,000 samples passed validation out of 17,538 generation attempts, resulting in an overall pass rate of 28.5%. Each accepted sample also requires approximately 8.4 LLM calls on average. Could the authors report the total generation time and cost, and compare them with other data synthesis methods at the same scale?
  6. Are the results on small-sample buckets sufficient to support the complexity analysis? Some complexity buckets in Table 4 contain only 2 or 12 test samples. A small number of samples can lead to large changes in accuracy. Therefore, the current results may not be sufficient to reliably compare performance improvements across different complexity regions.
  7. What evidence supports the claim that simply increasing the amount of training data leads to diminishing returns? The paper uses this claim as a key motivation for targeted synthesis. Please identify results in the cited work that directly support this claim, or provide other evidence supporting it.
  8. Could the authors provide a complete synthesis example? The example should include the seed question and SQL, extracted skeleton, target Depth–Width bucket and motifs, generated question–SQL pair, final complexity labels, and validation results. This would make the synthesis process and its controllability easier to understand.
2 comments

Thank you for the detailed questions. The completed audit quantifies target adherence and pair fidelity, and the controller comparison tests whether that targeting improves performance.

1. Target adherence and selection policy. The audit finds 90.0% exact agreement between target and final D/W values, with 10.0% target misses. Generation therefore provides soft targeting with measured adherence. We will state this explicitly in Section 4.2 and report exact agreement separately from lower-bound attainment and High coverage. Final detected motifs will be distinguished from recorded injection operators.

The target proposal probabilities, retry policy, mismatch handling, and original/synthetic/mixed bucket counts remain to be specified in the reproducibility description. The 90.0% agreement rate quantifies adherence; the proposal policy requires its own configuration record. We will keep accepted frequencies and proposal probabilities distinct.

2. Bucket sampling equals uniform sampling over the mixed pool. For the rule stated in Section 4.4, P(x)=(n_b/N)(1/n_b)=1/N. Your calculation is correct. The structural shift comes from changing the composition of the mixed training pool. We will revise Section 4.4 to describe that distributional effect accurately and remove the attribution of an extra reweighting benefit to this sampler.

3. Redundancy is checked at the instance level. The department-filter example shows why database contents matter: a condition required by the question can leave the output unchanged on a particular instance. Our gate selects examples whose injected motifs affect execution on the available database. We will state that scope and explain how NULLs, duplicates, and value distributions influence the decision. Cross-instance equivalence and the gate's false-rejection rate remain unmeasured. The separate pair audit evaluates whether the question faithfully expresses the SQL.

4. Generation overhead. Appendix D reports 17,538 attempts yielding 5,000 accepted instances (28.5%), approximately 8.4 LLM calls and 10 database executions per accepted instance. These quantities describe the pipeline's acceptance rate and per-instance operation count. Token usage, elapsed time, monetary cost, and matched-scale cost efficiency require separate measurements and remain outside the reported cost evidence.

5. Pooled results support the complexity comparison. We will center the evaluation on Spider All N=1,034 and pooled High N=196, with the N=2/12 cells retained as descriptive breakdowns. In the matched controller experiment, High EX is 70.4 for D/W, 65.3 for motif-only, and 66.8 for AST-depth targeting. Thus the central comparison uses the pooled High subset and directly changes the controller. These are point estimates; run-to-run uncertainty remains to be characterized.

6. Fixed-budget motivation. We will replace the general diminishing-returns assertion with the question tested here: whether allocating a fixed synthesis/training budget toward under-covered structures improves accuracy. The matched controller results directly support this motivation.

7. Pair fidelity and a complete synthesis example. The manual audit finds 96/100 faithful pairs for D/W and 95/100 for motif-only. This establishes a basic quality check alongside the structural audit: both sets have high, comparable fidelity. We will present question-SQL fidelity separately from SQL executability and motif validation.

For reproducibility, we will add one complete seed-to-acceptance example containing the seed question/SQL, extracted skeleton, selected target and motifs, generated pair, final labels, gate outcomes, and training inclusion. This will connect the target policy and validation stages in a single trace.

Reviewer_H43mOpenReview ↗

Reply to author

Thank you for the author's response. Considering the novelty and impact of the paper, I will maintain my score.

Reviewer_2GpAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a Depth–Width representation of SQL complexity and seven structural motifs for targeted Text-to-SQL data synthesis. It generates queries from training examples and filters them through executability, motif-consistency, and redundancy checks before fine-tuning. Experiments on Spider-dev and BIRD-dev report execution-accuracy improvements concentrated in the proposed high-complexity subsets. The evaluation includes comparisons with synthesis baselines and an ablation of the validation stages.

Critical strengths
  1. A practically relevant approach to targeted data synthesis. The paper addresses the problem of generating structurally meaningful SQL examples rather than simply increasing data volume. The motif consistency and redundancy checks are useful attempts to improve synthetic data quality.
Critical weaknesses
  1. The central semantic complexity formulation is not sufficiently convincing. The Depth–Width definition appears inconsistent with the motivating example in Figure 1, where a single-SELECT query is assigned Depth=2. Moreover, the claimed semantic equivalence of the illustrated SQL rewrites depends on unstated assumptions, and execution equivalence on a single database does not guarantee semantic equivalence in general. These issues weaken the paper's central conceptual contribution.
  2. Insufficient experimental validation and comparison with relevant baselines. The experiments do not adequately isolate the contribution of Depth–Width targeting from the overall synthesis and validation pipeline. Comparisons with simpler complexity measures and more recent structure-aware synthesis methods(particularly work from 2026) are missing. The limited model coverage and relatively small high-complexity subsets further reduce confidence in the generality of the reported improvements.
Questions for authors
  1. Why does Figure 1 assign Depth=2 to a single-SELECT LEFT JOIN query under the scope-dependency definition in Appendix A? What assumptions establish its equivalence to the NOT IN query?
  2. How is semantic equivalence verified beyond matching execution results on a single database? How do the authors account for NULL semantics and other database-dependent behaviors?
  3. How does Depth–Width targeting compare with simpler alternatives, such as AST depth, component count, or random motif targeting, under identical synthesis and training settings?
  4. Could the authors include more recent structure-aware Text-to-SQL synthesis methods, particularly relevant 2026 approaches, under comparable model and data budgets?
  5. Which data splits were used to select the core motifs and define complexity thresholds? Could the authors also clarify the relationship between the reported 7k-sample comparison budget and the 5,000 validated synthetic instances?
1 comment

Thank you for recognizing the practical value of targeted synthesis. The new experiments show that D/W adds predictive information and improves High accuracy under matched synthesis and training settings. We first address the definition issue.

1. D/W definition and Figure 1. Under Section 3.2 and Appendix A, the nested query has D=2 and the single-SELECT LEFT JOIN has D=1. Figure 1 is therefore incorrect, and we will replace it. The rewrite requires non-NULL compared names on both sides under the same comparison semantics. Our SQLite check provides a counterexample: with rows ('A',2017) and (NULL,2000), NOT IN returns no rows while LEFT JOIN returns A. We will state these conditions explicitly and align the example with the scope-based definition. The scorer used for the reported tables still needs to be reconciled with that definition.

2. D/W targeting improves High accuracy. We completed a three-arm comparison using 7B LoRA on Spider, holding the generator, seed distribution, motif quotas, shared filters, accepted-data size, and training updates fixed:

SettingAll EXHigh EXLow EX
D/W + motif81.270.483.8
Motif-only80.565.384.0
AST-depth + motif81.066.884.4

D/W raises High EX by 5.1 percentage points over motif-only and 3.6 points over AST-depth targeting, while retaining competitive overall EX. Low EX shows a small allocation trade-off. This comparison isolates the target controller within the shared pipeline; the original external comparison evaluates whole systems, and the validation ablation evaluates filters. We will add this controller comparison to clarify the contribution of D/W targeting.

3. D/W provides complementary difficulty information. Using fixed per-query model failures and database-grouped held-out evaluation, AUROC is 0.65 for AST depth, 0.63 for component count, 0.67 for Spider hardness, and 0.71 for D/W. Adding D/W to the combined conventional features raises AUROC from 0.68 to 0.72. These results support both its standalone predictive value and its complementarity to established measures. Together with the controller ablation, they validate the coordinate as a diagnostic and synthesis-control signal.

4. Semantic anchoring and execution checks. D/W describes semantically anchored structural complexity: scope dependencies determine D, and within-scope coupling determines W. Section 3.2 reports unchanged D/W on 87.3% of 2,847 output-equivalent rewrite pairs, providing empirical stability evidence for that collection. The redundancy gate tests whether an injected motif changes execution on the available database. Its evidence is instance-specific; NULLs, duplicates, and data distributions can produce accidental output equality. We will describe the gate at this level and reserve logical equivalence for rewrites with established conditions.

5. Related work and reproducibility. We will expand the method-level discussion of Text2SQL-Flow and RingSQL; matched-budget training comparisons with these methods remain outside the present experiments. The evaluated setting covers Qwen2.5-Coder 3B/7B on SQLite benchmarks. Section 4.2 specifies training-split generation seeds. The selection splits for vocabulary, risk, and thresholds, and the original/synthetic composition of Table 5's 7k budget relative to Appendix D's 5,000 accepted instances, remain to be documented. The new experiments address controller utility and difficulty prediction; these provenance details remain a separate reproducibility issue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.