acceptodds
Under review as a conference paper at ICLR 2027

Reasoning and Learning in Text-to-SQL: From Template Imitation to Strategy Recombination

Abstract

Large language models increasingly use chain-of-thought (CoT) reasoning to translate natural-language questions into SQL queries. Existing methods provide diverse reasoning templates, but how models use these templates after supervised fine-tuning and reinforcement learning remains unclear. In particular, can learning from multiple templates lead to useful reasoning beyond the imitation of a single template? In this work, we empirically study this transition on BIRD using two 7B models. We construct matched training data with balanced coverage of four templates and compare supervised fine-tuning, rejection fine-tuning, and subsequent GRPO. By analyzing freely generated responses, intervening on requested templates, and comparing reasoning functions before and after RL, we find that balanced training does not produce balanced template use. Instead, RL changes both the dominant forms of expression and the organization of reasoning functions within complete responses. Among same-question greedy responses with identifiable reasoning functions, RL changes their combination or order in 78.74% and 73.11% of cases for the two SFT models. Reasoning organizations associated with successful solutions become more frequent after RL across all four supervised training paths, including when these organizations are identified using pre-RL responses alone. These findings show how multi-template learning and RL can support a shift from template imitation toward task-useful CoT organization, providing an empirical basis for studying strategy recombination in text-to-SQL.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Text-to-SQL, Large Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ydUKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how multi-template learning and subsequent reinforcement learning affect reasoning behavior in text-to-SQL. Using two 7B backbones on BIRD, it compares template-balanced SFT/RFT and GRPO. The analysis incorporated template usage statistics, intervention of requested templates, reasoning functional profiles of full responses, held-out candidate evaluation. Among the four paths from supervision to GRPO, the greedy accuracy rate improved, while Pass@32 decreased. The authors interpret these findings as behavioral evidence of a shift from whole-template imitation toward task-useful reasoning organization.

Critical strengths
  1. The comparisons are transparent by setting matched training questions, fixed final checkpoints, shared evaluation and retention of failures.
  2. Using separate candidates for discovery and validation and pre-RL-only profile selection reduces selection bias in the utility analysis.
Critical weaknesses
  1. Pretrained models can already favor certain templates, so equal training counts do not guarantee equal use in greedy outputs. GRPO is also expected to make rewarded answers more likely. Given earlier findings on style concentration and answer coverage, the paper needs to explain more clearly what new insight it adds.
  2. It is unclear whether the model is actually combining reasoning steps more flexibly, or just switching templates or changing its wording.
Questions for authors
  1. What do we learn from this analysis beyond the expected effects of GRPO and the findings already reported by [1] and [2]?
  2. How can you tell whether the model is combining reasoning steps more flexibly, rather than just switching templates or changing its wording?

[1] Long Li et al. The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward. ICLR, 2026. https://arxiv.org/abs/2509.07430
[2] Yang Yue et al. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? arXiv:2504.13837, 2025. https://arxiv.org/abs/2504.13837

Reviewer_5oQBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies what happens to chain-of-thought when a 7B text-to-SQL model is trained on multiple reasoning templates and then reinforced with GRPO. Using matched data — 5,424 BIRD questions, each paired with an execution-verified CoT under four different templates — the authors compare supervised fine-tuning, rejection fine-tuning, and subsequent RL across two backbones, and track eight reasoning "functions" with a rule-based detector.

Critical strengths
  1. The experimental design is genuinely careful. Matched questions, matched template counts, same teacher filter, dev set excluded from training and rejection filtering and checkpoint selection, two backbones, one seed, predetermined final checkpoints. Separating SFT (32B teacher targets) from RFT (own execution-verified samples) cleanly isolates target construction from the optimization objective. Most text-to-SQL papers don't control this tightly.

  2. The reliability-vs-coverage framing is the paper's most useful contribution. Greedy EX and Pass@32 move in opposite directions after GRPO, consistently across both 7B models and reproduced in the 32B supplement. That's a practical warning for anyone doing RLVR on SQL: you're sharpening, not expanding. It also extends Yue et al. into a new domain with a cleaner multi-template setup.

  3. The intervention suite is creative and the pre-RL control is a real effort at non-circularity. Prescribed scaffolds, label perturbations (canonical/removed/renamed/mismatched), cross-question prefix donors, block reordering, and even/odd split-candidate selection with length matching. Selecting organizations from pre-RL answers only, then checking frequency on held-out post-RL candidates, is exactly the right instinct.

Critical weaknesses
  1. Everything load-bearing rests on an unvalidated regex detector. The eight "reasoning functions" come from fixed phrase rules requiring a schema identifier within 120 characters and six words. There's no human annotation, no inter-annotator agreement, no LLM-judge cross-check, no error analysis. If the detector is noisy or gameable, "strategy recombination" is a measurement artifact. For a paper whose entire claim is about organization, this needs way more validation.

  2. The effect sizes are tiny and the significance is shaky. The core Table 3 result is a +2–4% frequency shift, and held-out gaps are computed on subsets of 627/255/587/194 questions out of 1,534. Meanwhile 28/32 scaffold intervals, 18/24 marker contrasts, and 23/24 label contrasts include zero. Worse: pre-RL cohorts already show positive gaps (8.14%/1.83%/8.85%/3.42%), so it's unclear RL adds anything beyond selecting for what was already working.

Questions for authors

see weakness

Reviewer_E69TOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on a relevant learning question on whether exposure to diverse reasoning demonstrations gives a model useful flexibility after reinforcement learning, or mainly change which familiar responses it favors. The contribution focuses on linking observable reasoning organization to execution outcomes. The evidence supports redistribution toward some previously useful organizations, especially after teacher-based supervision. However, it is less conclusive about the validity and generality of ‘strategy recombination’ as the explanation.

The most reasonable contribution is a behavioral account of how post-training redistributes expressed organizations of reasoning. This can be informative even if the organizations are already present before RL. However, a frequency change is not itself evidence of learning reusable components or combining them in a new problem-solving procedure. The authors explicitly define recombination behaviorally, which avoids requiring an internal-mechanism claim. The remaining scientific question is whether the measured profiles capture a meaningful strategy-level phenomenon beyond altered wording, compression, and selection among familiar successful responses.

Critical strengths
  1. The study connects learning behavior to task outcomes under a reasonable experimental design. Matching training questions and template counts, separating teacher-based SFT from self-generated RFT, and evaluating all model states under common decoding rules make the comparisons informative. There is a sensible focus on eliciting greedy accuracy, finite-sample coverage, surface formats, and full-response profiles to expose different consequences of training.

  2. The utility analysis goes beyond retrospective selection of favorable examples. Splitting candidates into fitting and evaluation halves, selecting profiles from pre-RL answers alone, matching response lengths within questions, and reporting ordered and unordered analyses address important alternative explanations. In particular, the SFT results provide credible evidence that previously success-associated organizations become more common and retain a held-out accuracy advantage.

Critical weaknesses
  1. The analysis does not sufficiently distinguish changes in reasoning from changes in its expression. The paper documents changes in generated explanations, but it remains unclear how reliably these reflect changes in problem-solving strategies rather than presentation, brevity, or wording. The central measures therefore need stronger validation to support the interpretation as reasoning reorganization. This concern leaves the execution results intact but limits what they establish about how the models reason.

  2. The broader learning claim exceeds the demonstrated scope. The findings support changes in how the evaluated models use familiar reasoning patterns, but provide less evidence for a general transition toward flexible strategy recombination. The role of diverse-template supervision is not isolated, and the model and task coverage provides a limited basis for generalization. The contribution make more sense as a controlled behavioral study instead of an explanation of how models acquire reusable reasoning strategies.

Questions for authors
  1. What explanatory content does “strategy recombination” add beyond redistribution among familiar successful response organizations? Which results distinguish these accounts, and is the intended contribution a behavioral characterization or an explanation of how RL changes the solution process? The behavioral definition is explicit but the issue is what deeper conclusion the evidence supports.

  2. Which main claims are intended to generalize beyond the two Qwen2.5 backbones on BIRD, and why are the selected model variants, task, and four template sources sufficient to test those claims? Please distinguish the scientific rationale for these choices from the well-described procedures for constructing cohorts.

  3. What practical training decision does the organization analysis inform? SFT gives stronger individual answers in both backbones, while broader sampled coverage favors RFT only in the Coder comparison. Does the analysis guide the construction of supervision, or mainly explain behavior after training?

  4. How accurately and consistently do the eight-function rules identify the intended reasoning functions across SFT, RFT, and GRPO?

  5. How should the account of task-useful organization differ between SFT and RFT? Which claims are supported across all training paths, and which should be restricted to the SFT evidence?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.