acceptodds
Under review as a conference paper at ICLR 2027

Enhancing Robustness of Small-Scale Models for Text-to-SQL via Semantic Enhancement and Noise-Aware Methods

Abstract

With the advancement of large language model technology, Text-to-SQL techniques have achieved significant progress and are widely applied in scenarios such as search engines. However, current small-scale models still face challenges, particularly poor robustness, especially when the database schema or user natural language queries lack clear semantic descriptions, resulting in decreased SQL generation accuracy. Although large-scale models demonstrate stronger robustness and higher accuracy, their high computational and deployment costs limit their use in private deployment settings. To address these issues, this paper proposes a reverse semantic enhancement method that leverages prior knowledge such as existing database values to enrich the semantic information of the database schema and user queries. Furthermore, to mitigate the noise introduced during semantic enhancement, a noise-aware training approach is designed to critique noisy data and identify effective semantic information, thereby improving the model’s resistance to interference. Extensive experiments on the Spider and BIRD datasets demonstrate the effectiveness of the proposed methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Models, Text-to-SQL, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_86mROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the robustness of small-scale Text-to-SQL models when database schemas and user questions provide insufficient semantic information. It proposes a two-stage framework. First, a large model uses database values, historical queries, and training-set gold SQL to synthesize schema descriptions, entity mappings, and sub-question hints, which are distilled into SynGen. Second, a downstream Noisy-SQL model is trained using noise injection and GRPO with an additional reward for identifying and correcting misleading hints. Experiments cover Spider, BIRD, robustness variants, and multiple model sizes. The appendix additionally provides cryptic-schema evaluation, human validation, and three-seed results.

Critical strengths

The paper focuses on the difficulties that small-scale Text-to-SQL models face when interpreting ambiguous database schemas and bridging the semantic gap between schemas and user questions. The authors use database example values, historical queries, and annotated training SQL to construct training data containing schema comments, entity-level hints, and sub-question-level hints through reverse semantic inference. These synthesized annotations serve as supervision for SynGen, which learns to generate auxiliary information intended to help the downstream model understand database schemas and their correspondence with user questions. A positive aspect is that the authors do not assume these enhancements are always reliable; they also consider how incorrect hints may interfere with SQL generation. To address this issue, they inject semantically similar but incorrect hints into the enhanced training data and incorporate a noise-awareness reward into GRPO training, encouraging the downstream model to identify and correct misleading information.

Critical weaknesses

W1. Missing controlled comparisons leave the independent contribution of reverse semantic inference insufficiently established. Appendix D.1 demonstrates improvements from adding the generated semantic information to other downstream systems, while Appendix D.3 compares different enhancers and teacher models. These results support the effectiveness of semantic enhancement and training SynGen. However, they do not sufficiently establish whether the gains arise specifically from the proposed reverse semantic inference, rather than general task-specific supervision and knowledge distillation. W2. The noise-aware reward assigns erroneous high scores to failed trajectories, weakening the interpretation of the training mechanism. The noise is deliberately constructed by the authors, and reward computation explicitly uses Noise_Labels. The model could therefore be required to output explicit noise predictions, enabling a verifiable reward computed directly against the known labels. The authors motivate introducing a Qwen2.5-72B judge by the diversity of free-form reasoning trajectories, but do not establish its advantage over a label-based reward. Furthermore, Figure 10 shows that 22 of the 24 trajectories judged by humans as failing to identify noise receive the judge’s maximum score, revealing substantial erroneous rewards on critical failure cases.

Questions for authors

1.What justifies the two-stage design? Could semantic enhancement and SQL generation be combined into a single model with one generation pass? How would accuracy, robustness, and end-to-end latency change relative to the current SynGen→Noisy-SQL pipeline? Do these results establish the need for a separate enhancer? 2.What practical deployment advantage does the small-model cascade offer over using the teacher directly? Could the authors compare the complete cascade against direct SQL generation by the teacher in terms of accuracy, end-to-end latency, total generated tokens, peak memory, and deployment cost, with hardware and inference settings clearly specified? Does the cost accounting include both SynGen and Noisy-SQL? Under what resource constraints and accuracy requirements does the cascade offer a more useful performance–cost trade-off? 3.Does the benefit of semantic enhancement depend on the downstream model’s capability? Could the authors compare the same strong SQL generator with and without SynGen-generated enhancements, and contrast its gains with those observed for small models? If such evidence is unavailable, should the conclusions about the enhancement strategy’s effectiveness be explicitly limited to the tested small-model settings? 4.Why is an LLM judge needed when noise labels are already known? Could the authors compare the current judge-based reward against a verifiable reward that directly scores explicit noise predictions against known labels, using identical noisy data, the same underlying rewards, and comparable training budgets? Furthermore, what causes the judge to assign maximum scores to 22 of the 24 human-rated failures in Figure 10, and how do these erroneous rewards affect training? While Table 6 supports the empirical utility of the noise reward, what additional evidence establishes that the benefits of an LLM judge justify its extra cost and reliability risks?

Reviewer_XSGqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework to improve the robustness of small-scale Text-to-SQL models through semantic enhancement and noise-aware training. The method first uses a large language model to synthesize schema descriptions and question-level hints from database values and historical question–SQL pairs. These annotations are then used to train a smaller semantic generator (SynGen). A downstream Text-to-SQL model is further optimized using GRPO with a noise-aware reward to identify and correct misleading semantic hints.

Critical strengths

S1. The paper addresses a practical issue in Text-to-SQL: the difficulty of handling ambiguous schema descriptions and natural-language questions with small models.

S2. The proposed method achieves some improvements on robustness-focused benchmarks, particularly Spider-Syn. The paper also includes component ablations, evaluations across model sizes, and multi-seed experiments.

Critical weaknesses

W1. Limited technical novelty. The proposed framework largely combines established techniques, including LLM-based semantic data augmentation, supervised fine-tuning, hard-negative noise injection, and GRPO-based training.

W2. Several components are closely related to existing work. For example, SEA-SQL has explored semantic schema enhancement, while JOLT-SQL addresses robustness through confusion-aware noisy schema sampling. Although the combination of SQL-grounded hint synthesis and noise-aware reinforcement learning is somewhat different, the paper does not clearly establish what new methodological insight or capability this combination provides.

W3. The result is somewhat weak in current benchmark leadboard.

W4. Limited evidence of generalization to newer and more realistic benchmark, such as spider2.0.

Questions for authors

None.

Reviewer_FXUJOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper targets the robustness of small (1.5–3B) Text-to-SQL models when schemas or questions are semantically poor (cryptic column names, synonyms). First, Qwen2.5-72B performs "reverse semantic inference" from database values, historical NL–SQL pairs and gold SQL to generate schema comments and entity- and sub-question-level hints, which are distilled into a 3B generator (SynGen). Second, a 3B SQL model (Noisy-SQL) is trained with GRPO on enhanced inputs containing injected similar-but-wrong hints. Its reward combines execution, format, and a noise-handling score from a 72B LLM judge. With Qwen2.5-Coder-3B, the approach reports 83.6%/79.9% execution accuracy on Spider-Syn/Spider-Realistic and 60.5% on BIRD-dev, with ablations, cross-backbone tests and a plug-in study on other systems.

Critical strengths
  • Robust Text-to-SQL for small, privately deployable models is a real need. Using database values and query logs as cheap prior knowledge is a sensible, deployable idea.
  • Coherent two-part design: Pairing semantic enhancement with explicit training to detect and correct wrong hints addresses the obvious downside of automatic enrichment. The noise-similarity threshold study (0.7/0.9/0.99) supports the hard-negative design.
  • Consistent component gains: Schema enhancement, question enhancement and noise-aware training each help across BIRD and the Spider variants, on two backbones (Qwen2.5 1.5B/3B, Llama-3.2-3B).
  • Plugging the generated semantics into SEA-SQL, CodeS and JOLT, plus the SNAILS cryptic-schema test and human grading of synthesized hints, broaden the evidence beyond the main tables.
Critical weaknesses
  • Inconsistent headline results: The text attributes 83.6/79.9 on Spider-Syn/Real to the Spider-trained model, but the conclusion claims SOTA with 81.4/77.4. The ablation's "full" row equals the BIRD-trained model. Table 1 has missing or misaligned cells and a wrong citation for DIN-SQL. As a result, the main claim cannot be verified reliably.

  • Training relies on Qwen2.5-72B for data synthesis and as the GRPO reward judge (2,560 judge calls per step). Inference runs two 3B models in sequence. Comparisons against 7B baselines trained without a 72B teacher therefore do not isolate the method's contribution from distillation.

  • Possible information leakage at inference: Schema enrichment uses historical NL–SQL pairs for the target database. It is unclear whether, for Spider-dev/test and BIRD-dev, these pairs come from the evaluation databases or questions, which would inflate results.

  • Marginal gains and circular robustness test: Noise-aware training adds ~1–2.4 points over enhancement alone, reported from a primary run (multi-seed only in the appendix). The noise-robustness analysis (Figure 5) injects noise with the same procedure used in training, so it measures in-distribution resilience.

  • Limited robustness coverage and SOTA claims: Only Spider-Syn and Spider-Realistic are used, although Dr.Spider is cited. On BIRD-dev and Spider-test, Omni-SQL-7B (63.9 / 87.9) outperforms Noisy-SQL, so "SOTA" holds only on the two synonym-style sets that the entity hints directly target.

Questions for authors
  1. Which Noisy-SQL model (BIRD- or Spider-trained) produced each number in Tables 1, 2, 4 and 5 and the conclusion? Please provide a corrected, complete Table 1.

  2. At inference on each evaluation set, which "historical query pairs" and database values does SynGen see? Are any drawn from the evaluation split's databases or questions?

  3. Does the latency in Table 4 include SynGen's generation time? What is the total parameter count and latency of the two-model pipeline compared with a single 7B model?

  4. What is the performance of a 3B GRPO baseline trained on the same 72B-synthesized enhanced data without noise injection or the noise reward? This would isolate noise-aware training from distillation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.