acceptodds
Under review as a conference paper at ICLR 2027

SQL-Structure-Aware On-Policy Distillation for Text-to-SQL

Abstract

On-policy distillation (OPD) provides token-level teacher feedback for training Text-to-SQL models. However, uniform weighting allows lengthy reasoning text to dominate the distillation objective and does not account for how tokens jointly express SQL operations, columns, and values. We propose a SQL-structure-aware weighting method for OPD that controls the total weight assigned to final SQL and reasoning steps and adjusts local weights through clause comparisons. The method uses abstract syntax trees to compare student-generated clauses with the reference SQL and other execution-correct samples, accounting for their query context. It then propagates SQL weights to the corresponding reasoning steps through aligned SQL fragments. These weights scale the existing teacher feedback without changing the task reward or adding inference-time procedures. Trained only on BIRD with a Qwen3.5-4B student, our method achieves 64.64% execution accuracy on BIRD-dev, improving over vanilla OPD by 3.58 percentage points. It also improves execution accuracy across five Spider benchmarks by 1.75–2.61 points.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-SQL, On-Policy Distillation, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xChWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed method try to combine the benefit of RLVR and OPD by re-weighting on SQL part generation. The re-weighting includes three steps 1/ assigns predefined shares of the total distillation weight to final SQL and alignable reasoning steps 2/ it uses ASTs to parse the SQL structure 3/ Step–SQL Fragment protocol pairs are extrated to align reasoning step with SQL generation. From the result, it shows the proposed method outperforms OPD in BIRD and Spider benchmark from 1.75 to 3.58 in EX.

Critical strengths

1/ The idea of reweighting the SQL part and aligned the reasoning part better is reasonable. 2/ The performance gain compared to OPD makes sense.

Critical weaknesses

The experiment is not comprehensive: 1/ No teacher model perrformance. 2/ Only one student model is tested. 3/ No RLVR performance. 4/ the ablation of Table 2, only test BIRD. It should covered Spider as well.

Minor: 1/ the illustrating examples in Figure 1 and 2 are different and redundant 2/ line 338, the Claude Sonnet-3.7 should be model instead of Method

Questions for authors

N/a

3 comments

We thank the reviewer for the constructive comments and suggestions. Below, we address each concern point by point.


1. Teacher Model Performance and Additional Student Model Size

1/ No teacher model performance. 2/ Only one student model is tested.

We actually evaluated and reported both the Qwen3.8-27B teacher model and an additional Qwen3.5-9B student model in Section 4.8 (Effectiveness Across Student Model Sizes) and Appendix E (Figure 4) of the manuscript:

  • Teacher performance: The Qwen3.8-27B teacher achieves 63.08% execution accuracy (EX) on BIRD-dev.
  • Additional student size (Qwen3.5-9B): When using Qwen3.5-9B as the student model, our method improves BIRD-dev EX from 56.29% (before training) and 61.64% (vanilla OPD) to 65.04% (+3.40 percentage points over vanilla OPD). This gain is consistent with the improvement observed on the Qwen3.5-4B student (from 48.92% before training and 61.06% with vanilla OPD to 64.64%, +3.58 points).

In the revised manuscript, we will also include the 27B teacher and the 9B student results directly in the main results table (Table 1) to make these comparisons immediately visible.


2. RLVR (GRPO-Only) Performance

3/ No RLVR performance.

Following your suggestion, we added a controlled GRPO-only baseline, while keeping the exact same Qwen3.5-4B student, training data, and task reward setup). As shown in the table below:

  • vanilla OPD vs. GRPO-only: Incorporating teacher distillation in vanilla OPD improves execution accuracy over GRPO-only by 1.05 percentage points on BIRD-dev (61.06% vs. 60.01%) and by 0.78–1.57 percentage points across the five Spider benchmarks, confirming that token-level teacher feedback from OPD provides helpful guidance beyond execution-based RLVR alone.
  • Ours vs. vanilla OPD & GRPO-only: By further optimizing distillation weight allocation across SQL regions, clauses, and aligned reasoning steps, our method achieves substantial additional gains across all six benchmarks (+4.63 points over GRPO-only and +3.58 points over vanilla OPD on BIRD-dev), validating the effectiveness of our SQL-structure-aware weighting framework.
MethodBIRD-devSpider-devSpider-testSpider-SynSpider-DKSpider-Real
GRPO-only (RLVR)60.0182.0180.2579.7975.1479.53
vanilla OPD61.0682.7981.4681.3376.6481.10
Ours64.6484.8283.5683.0879.2583.27

3. Ablation Study on Spider

4/ the ablation of Table 2, only test BIRD. It should covered Spider as well.

We conducted an additional ablation study trained on the Spider training split and evaluated on both Spider-dev and Spider-test across configurations A0–A4. As shown in the table below, our method remains consistently effective on Spider:

  • Adding explicit region weighting (A1) improves Spider-dev EX from 84.91% to 86.27% (+1.36 points) and Spider-test EX from 85.61% to 86.54% (+0.93 points).
  • Incorporating AST clause comparison (A3) further raises Spider-dev EX to 86.75% (+1.84 points over A0) and Spider-test EX to 86.96% (+1.35 points over A0).
  • The complete method (A4) with step-level weight propagation achieves the best performance on both splits, reaching 87.04% on Spider-dev (+2.13 points over A0) and 87.10% on Spider-test (+1.49 points over A0).
Config.Region weightingWithin-region weightingAST clause comparisonWeight propagationSpider-dev EX (%)ΔSpider-test EX (%)Δ
A0--------84.91--85.61--
A1✓------86.27+1.3686.54+0.93
A2✓✓----85.88+0.9786.63+1.02
A3✓✓✓--86.75+1.8486.96+1.35
A4✓✓✓✓87.04+2.1387.10+1.49

Note: ✓/-- denotes enabled/disabled. A0 is vanilla OPD; A4 is the complete method. Δ denotes the change relative to A0 in percentage points.


4. Minor Issues

Thank you for pointing out these presentation details:

  1. Unifying Figures 1 and 2: In the revised manuscript, we will unify the illustrating example across Figure 1 and Figure 2 so that both figures trace the exact same query and error case, reducing cognitive load for readers.
  2. Table 1 formatting (Line 338): We will revise the row for Claude Sonnet-3.7 in Table 1 and carefully proofread all tables and notations throughout the paper.

Dear Reviewer xChW,

As the discussion period comes to an end soon, we wanted to gently follow up to see if our responses have addressed your concerns.

Specifically, we clarified the misunderstanding regarding Points 1 and 2 (pointing to the 27B teacher and additional 9B student results in Section 4.8 and Appendix E, which we will move directly into Table 1 for immediate visibility), added the controlled RLVR (GRPO-only) experiments, and conducted the full A0–A4 ablation study on Spider. We guarantee that we will carefully incorporate every point of your feedback—including these experiments and the figure/table fixes—point by point into our revised manuscript.

We would be glad to discuss any remaining questions before the deadline, and if your concerns are resolved, we kindly hope you will reconsider our work. Thank you again for your valuable feedback!

Best regards,

Thanks for the pointing the experiments result in Section 4.8 and Appendix E. I would keep my current score of 2 (from 1.8 -> 2.4).

Reviewer_XanmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a SQL-structure-aware token weighting framework for on-policy distillation (OPD) in Text-to-SQL training, addressing two limitations of standard uniform-weight OPD when combined with execution-based reinforcement learning (GRPO): lengthy chain-of-thought reasoning traces dominating the distillation gradient over short SQL queries, and independent token-level teacher feedback ignoring clause-level structural context. Without modifying the task reward, teacher scoring direction, or single-pass inference procedure, the authors introduce a three-part non-negative reweighting scheme that (1) parses student-generated SQL into Abstract Syntax Trees (ASTs) to compare normalized clauses and structural skeletons against both the ground-truth reference SQL and execution-correct peer rollouts within matching dependency contexts, (2) propagates these local clause weights backward to corresponding natural-language explanations and SQL fragments via a structured Step–SQL Fragment reasoning protocol, and (3) normalizes global distillation shares so that the final SQL and aligned reasoning steps receive fixed proportions of the total weight (75% and 25%, respectively). The method is empirically validated by training Qwen3.5-4B and Qwen3.5-9B student models with a frozen Qwen3.8-27B teacher exclusively on the BIRD training set, where the 4B student reaches 64.64% execution accuracy on BIRD-Dev (+3.58 percentage points over vanilla OPD and surpassing the 63.08% teacher) and generalizes zero-shot to five Spider benchmarks (Spider-Dev, Spider-Test, Spider-Syn, Spider-DK, and Spider-Real) with gains of +1.75 to +2.61 percentage points, supported by progressive component ablations and hyperparameter sensitivity analyses.

Critical strengths
  1. Principled, zero-inference-overhead solution to length imbalance and structural credit assignment in On-Policy Distillation: In reasoning-augmented Text-to-SQL, standard uniform-weight OPD allows verbose chain-of-thought traces (often >95% of generated tokens) to dominate the distillation gradient over the short executable SQL query, while scoring SQL tokens without clause-level context. The proposed framework addresses both issues purely at training time by fixing global weight shares between the final SQL (75%) and aligned reasoning steps (25%), comparing normalized AST clauses against both the reference SQL and execution-correct peer rollouts (which naturally accommodates valid alternative SQL formulations), and propagating clause weights back to reasoning steps via the Step–SQL Fragment protocol. Why it is consequential: It achieves substantial accuracy gains (+3.58 points over vanilla OPD on BIRD-Dev with a compact 4B model) while preserving single-pass greedy decoding at inference time—avoiding the heavy latency and token costs of multi-turn agentic repair pipelines.

  2. Consistent zero-shot cross-benchmark generalization and student-surpassing-teacher performance in controlled ablations: Trained exclusively on the BIRD training set, the Qwen3.5-4B (64.64% EX) and Qwen3.5-9B (65.04% EX) students outperform vanilla OPD by +3.58 and +3.40 percentage points on BIRD-Dev, surpassing their frozen Qwen3.8-27B teacher (63.08% EX) when paired with execution-based GRPO—and transfer zero-shot across five unseen Spider benchmarks (Spider-Dev, Spider-Test, Spider-Syn, Spider-DK, and Spider-Real) with uniform gains of +1.75 to +2.61 points. Moreover, the progressive component ablation (Table 2) clearly isolates the positive contributions of global region balancing (+1.50%), AST clause comparison (+0.98%), and step weight propagation (+1.30%). Why it is consequential: The consistent out-of-domain transfer across five benchmarks under identical training budgets demonstrates that structure-aware OPD genuinely improves the student's generalizable SQL reasoning capabilities rather than overfitting to benchmark-specific artifacts.

Critical weaknesses
  1. Counter-intuitive downweighting of erroneous clauses dampens teacher error correction: Under Equation (9) and Figure 2(d), when a student generates an incorrect clause (q_c = 0, e.g., WHERE year=2012), it is assigned a smaller weight (u_c = 0.5), which cuts the teacher's negative corrective gradient in half (from -0.80 to -0.40), whereas already-correct clauses (q_c = 1) are upweighted (u_c = 1.5). Why it is consequential: Dampening the teacher’s penalty on the exact clauses where the student makes mistakes contradicts the intuition of error correction, and the authors never ablate the opposite direction (upweighting errors) or separate weights for positive vs. negative teacher advantages.

  2. Direct mismatch between the headline motivation (Figure 1b) and the AST comparator (Equation 6): Figure 1(b) motivates clause-level AST weighting using a missing SUM aggregation (SELECT num_students vs. SELECT SUM(num_students)). However, as admitted on Line 228, because SELECT n and SELECT SUM(n) have different operator skeletons (η_c ≠ η_(c′)), Equation (6) returns (unknown) and leaves the clause weight at the neutral value 1.0. Why it is consequential: The proposed AST comparison rule is overly restrictive, requiring identical operator skeletons to flag a disagreement (r=0), and consequently fails to activate on the very operator-omission error used to motivate the paper.

  3. Missing GRPO-only ablation (λ_(OPD) = 0) and omission of recent RL/OPD Text-to-SQL baselines: The training loss combines execution-based GRPO (which includes a custom format reward q_(fmt) enforcing the Step–SQL Fragment protocol) with OPD, yet the paper never evaluates a controlled GRPO-only baseline without teacher distillation, nor does it empirically compare against recent RLVR/OPD Text-to-SQL methods discussed in the introduction (e.g., SERL-SQL, Progressive RLVR, Reasoning-SQL, SQL-R1). Why it is consequential: Especially because the 4B and 9B students (64.64% and 65.04%) outperform their 27B teacher (63.08%), it is unclear how much of the gain over the base model comes from GRPO plus the structured Step–SQL format reward alone versus the proposed distillation weighting.

Questions for authors

It has been mentioned in the paper that the token_length for training was set to 8192, the question I have is how authors were bale to fit the long database schema and all output tokens within this budget?

3 comments

We thank the reviewer for the thorough analysis and insightful questions. Below, we address each concern concisely.


1. Downweighting Disagreeing Clauses vs. Reversing Weighting Direction (Weakness 1)

  • Theoretical trade-off under student-prefix conditioning: In OPD, the token advantage AtOPDA_t^{\mathrm{OPD}} reflects the teacher–student log-probability difference conditioned on the student's generated prefix y<ty_{<t}, rather than an oracle token-level correctness label. As noted at the end of Section 3.5, nonnegative weights adjust distillation budget allocation rather than construct local correctness rewards. Whenever a disagreeing clause contains both corrective negative signals (AtOPD<0A_t^{\mathrm{OPD}} < 0) and prefix-induced positive signals (AtOPD>0A_t^{\mathrm{OPD}} > 0), upweighting erroneous clauses acts as a double-edged sword: it amplifies negative error correction while simultaneously reinforcing erroneous tokens favored under the student's flawed context.
  • Empirical prefix-bias diagnostic on real training rollouts: Analyzing real training trajectories confirms that when the same erroneous decision appears earlier in the student's reasoning prefix (e.g., first writing WHERE year=2012 in reasoning and repeating it in Final SQL), the teacher may initially penalize the mistake in reasoning, but once conditioned on that flawed reasoning prefix, it becomes strongly biased toward continuing the error in Final SQL:
    • Retaining the full student prefix, the teacher ranks the erroneous token/function as Top-1 in 91.67% of cases, assigning it a mean conditional probability of 87.12%.
    • On the exact same queries and SQL, removing only the preceding erroneous reasoning prefix causes the teacher's Top-1 rate on the erroneous token to plummet from 91.67% to 16.67%, and its mean probability to drop from 87.12% to 14.83%.
  • Controlled reversed-direction ablation: Following your suggestion, we conducted a controlled ablation under identical training settings comparing our original direction (upweighting agreeing clauses and downweighting conflicting clauses) against the reversed direction (downweighting agreeing clauses and upweighting conflicting clauses) on BIRD-dev:
ConfigurationBIRD-dev EX (%)Δ
Original direction (Ours)64.64--
Reversed direction62.88−1.76

Upweighting conflicting clauses degrades BIRD-dev accuracy by 1.76 percentage points, validating our design choice.


2. Alignment Between Figure 1(b) Motivation and Equation (6) AST Comparator (Weakness 2)

We acknowledge that rule-based AST comparison has inherent coverage limitations (Section 5), as accurate rule-based AST matching across arbitrary SQL structures is challenging. Our setting is designed to avoid mistakenly downweighting valid equivalent SQL reformulations across different operator skeletons (e.g., COUNT(*) vs. COUNT(primary_key)), applying the strongest 0.50.5 downweighting only when the operator skeleton is identical while column names or literal values conflict (e.g., 2012 vs. 2011 in Figure 2).

We also apologize that our wording in Section 1 and Figure 1(b) caused confusion. Their purpose is to illustrate that local token overlap (e.g., matching SELECT) is insufficient, and distillation weighting must instead account for the complete clause (operations, columns, and values). Comparing operations in a complete clause directly determines whether the threshold for r=1r = 1 (σc=σc′\sigma_c = \sigma_{c'}) is met: only the complete SELECT SUM(num_students) receives r=1r = 1 (upweighted to 1+β1 + \beta), whereas omitting SUM forfeits this positive agreement support and leaves the clause at uSELECT=1.0u_{\mathrm{SELECT}} = 1.0. In the full query of Figure 1 (SELECT num_students FROM university_year WHERE year = 2011), the matching FROM and WHERE clauses receive r=1r = 1 (uc=1+βu_c = 1 + \beta). After fixed regional budget normalization in Equations (11)–(12), the final weight wtw_t on SELECT num_students is automatically reduced to 11+β\frac{1}{1+\beta} of the weight on supported clauses (such as WHERE year = 2011); meanwhile, through weight propagation in Equation (10), its aligned reasoning step likewise receives a lower relative weight within the reasoning budget. Thus, clause-level weight adjustment actively takes effect in the Figure 1 example. We will refine the presentation of this point in the revised manuscript.

3. Controlled GRPO-Only Baseline and Recent RL/OPD Baselines (Weakness 3)

  • Controlled GRPO-only ablation: We added a controlled GRPO-only baseline keeping the exact same Qwen3.5-4B student, training data, and Step–SQL format reward setup:
MethodBIRD-devSpider-devSpider-testSpider-SynSpider-DKSpider-Real
GRPO-only60.0182.0180.2579.7975.1479.53
vanilla OPD61.0682.7981.4681.3376.6481.10
Ours64.6484.8283.5683.0879.2583.27

Vanilla OPD improves over GRPO-only by +1.05 points on BIRD-dev and +0.78–1.57 points across Spider benchmarks, while our SQL-structure-aware weighting brings substantial further gains (+4.63 points over GRPO-only and +3.58 points over vanilla OPD on BIRD-dev), confirming that the improvement stems from our structured distillation weighting rather than GRPO + format reward alone.

  • Recent RL/OPD Text-to-SQL methods: Methods such as SERL-SQL operate in multi-turn interactive agent settings with execution feedback across turns, whereas our main setup evaluates single-turn generation without test-time database interaction. To verify compatibility under comparable multi-turn conditions, we reproduced SERL-SQL (68.58% EX) and integrated our token-level structural weighting into its hindsight OPD objective, reaching 69.95% EX (+1.37 points). We will include both the GRPO-only ablation and expanded RLVR/OPD comparisons in the revision.

4. Prompt and Output Token Budget (Question 1)

First, as noted in Appendix C, 8,192 tokens is the prompt limit alone, while generated outputs have a separate 4,096-token response limit (total context budget of 12,288 tokens). Second, we do not serialize entire database tables into the prompt; instead, each schema includes compact CREATE TABLE DDL definitions with at most 3 sampled example values per column (plus brief column descriptions when available). Under this representation, prompts do not exceed the 8,192-token length limit. We will clarify this schema serialization detail in Appendix C.

Dear Reviewer Xanm,

As the discussion period draws to a close, we would like to sincerely thank you for your constructive feedback and insightful suggestions, which have played a significant role in improving the quality of our work. We guarantee that we will carefully incorporate every point of your feedback—including the prefix-bias analysis, reversed-weighting and GRPO-only ablations, and presentation clarifications—point by point into our revised manuscript. Thank you again for your time and support!

Best regards,

Reviewer_egZgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a SQL-structure-aware weighting method for on-policy distillation in Text-to-SQL. The method adjusts distillation weights between reasoning steps and final SQL, uses AST-based clause comparison, and transfers SQL weights to corresponding reasoning steps. Experiments on BIRD and five Spider benchmarks show consistent improvements over vanilla OPD, supported by ablation studies and evaluations on different student model sizes.

Critical strengths
  1. The paper identifies a problem in vanilla OPD, where long reasoning text can receive much more training weight than the final SQL. In Section 3, the authors propose a SQL-structure-aware weighting method that combines region weighting, AST-based clause comparison, and weight propagation to reasoning steps. This provides a practical way to improve distillation without changing the inference procedure.
  2. Table 1 shows that the proposed method outperforms vanilla OPD on BIRD and all five Spider benchmarks, with a 3.58-point improvement on BIRD-dev. Table 2 also shows that AST-based clause comparison and reasoning weight propagation provide additional improvements beyond simple region weighting. These results support the effectiveness of the proposed method.
Critical weaknesses
  1. The proposed method mainly extends existing OPD training through token-level loss reweighting. While the SQL-specific design is reasonable, similar ideas have been explored in selective distillation and structure-aware Text-to-SQL training. In particular, the authors discuss SERL-SQL in Section 2 but do not include it in the experimental comparisons. This makes it difficult to assess how much the proposed method advances beyond existing approaches.
  2. In Section 3, the method requires each reasoning step to contain a natural language explanation and a corresponding SQL fragment. However, the paper does not clearly explain how this format is enforced or report how often the generated fragments can be successfully aligned with the final SQL. Since the same output protocol is used during inference, it may also introduce additional token and latency costs that are not evaluated. These missing details make it difficult to assess the practicality and general applicability of the method.
Questions for authors
  1. In Section 3, the proposed method combines several techniques that appear closely related to existing work. SERL-SQL [1] has explored selective teacher-guided weighting for Text-to-SQL training, while AST-Masking [2] uses SQL abstract syntax trees to assign structure-aware training weights. StepOPSD [3] also studies step-level redistribution of distillation signals, and Struct-SQL [4] explores structured reasoning for Text-to-SQL knowledge distillation. Although these methods differ in their objectives and implementations, several core ideas behind the proposed framework have already been investigated. Could the authors clarify which components are technically new compared with these methods? In particular, what is the main methodological contribution beyond combining existing weighting and structured reasoning techniques?
  2. In Section 2, the authors explicitly discuss SERL-SQL [1], which also uses teacher–student feedback to selectively adjust training signals for Text-to-SQL. However, SERL-SQL is not included in Table 1, despite being one of the most closely related methods. Could the authors explain why this baseline was excluded? Given the differences in training objectives and interaction settings, could the authors clarify whether a direct comparison is feasible and how the proposed method compares with SERL-SQL under comparable conditions?
  3. In Section 3.1, the authors state that each reasoning step consists of a natural language explanation and a corresponding SQL fragment. Section 3.4 further relies on matching these fragments to the final SQL to propagate distillation weights. However, the paper does not clearly explain how this structured output format is enforced. Is it specified through prompting, training rewards, or both? If prompting is used, could the authors provide the prompt template? Appendix C mentions a structure penalty based on step numbering, fragment presence, and correspondence to the final SQL. Could the authors clarify how this penalty interacts with the output protocol and report how frequently the generated fragments are successfully aligned? Also, Section 3.5 states that the same output protocol is used at inference time. Does requiring these additional SQL fragments increase generation length or inference latency compared with standard reasoning? These details would help assess the reproducibility and practical cost of the method.

[1] Liu, Tao, et al. "SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning." arXiv preprint arXiv:2608.00485 (2026). [2] Zhu, Xinyu, et al. "Structure-Aware NL-to-SQL for SFC Provisioning via AST-Masking Empowered Language Models." ICC 2026-IEEE International Conference on Communications. IEEE, 2026. [3] Zhang, Yanfei, Xu Lin, and Chenglin Wu. "StepOPSD: Step-aware online preference self-distillation for agent reinforcement learning." Findings of the Association for Computational Linguistics: EMNLP 2026. 2026. [4] Thaker, Khushboo, and Yony Bresler. "Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL." arXiv preprint arXiv:2512.17053 (2025).

2 comments

We thank the reviewer for the thoughtful feedback and constructive questions. Below, we address each point concisely.


1. Comparison and Integration with SERL-SQL (Weakness 1 & Question 2)

SERL-SQL [1] is designed for multi-turn interactive agents, using privileged execution feedback to apply step-level advantage weighting across interaction turns without addressing how SQL tokens are diluted by lengthy reasoning within a single response. Because our method targets single-turn generation without multi-turn database interaction at inference, a direct comparison in Table 1 is not apples-to-apples.

To evaluate compatibility under comparable conditions, we reproduced SERL-SQL and integrated our token-level weighting mechanism into its hindsight OPD objective:

MethodEX (%)Δ
SERL-SQL (reproduced baseline)68.58--
SERL-SQL + Ours (weighted hindsight OPD)69.95+1.37

Adding our token-level weighting improves SERL-SQL by +1.37 percentage points, demonstrating that our intra-response structural weighting is complementary to multi-turn selective distillation and remains effective in agentic settings.


2. Technical Contributions Beyond Related Work (Question 1)

Although [1–4] share high-level themes with our work, they differ fundamentally in problem settings, supervision signals, and mechanisms:

  1. Vs. AST-Masking [2]: Identical SQL semantics can be written in diverse surface forms. While AST-Masking assigns static weights solely from syntactic node types in a single ground-truth query, we resolve alias ambiguities and dynamically compare student clauses against both the reference SQL and execution-correct peer rollouts from the same group. This expands coverage of valid alternatives and improves generalization.
  2. Vs. SERL-SQL [1] and StepOPSD [3]: Both focus on turn/step-level advantage weighting in multi-turn interactive agents using privileged environment feedback. In contrast, we address the dilution of SQL distillation signals caused by lengthy reasoning text in single-turn Text-to-SQL, explicitly controlling regional weight budgets and propagating clause-comparison weights back to aligned reasoning steps.
  3. Vs. Struct-SQL [4]: Struct-SQL performs offline data synthesis, prompting a teacher to generate fixed query-plan traces followed by standard SFT with uniform token weights. Conversely, our method is an online OPD training framework that uses paired SQL fragments as anchors to transfer clause-level distillation weights to corresponding reasoning steps.

3. Step–SQL Fragment Enforcement, Alignment Rate, and Inference Cost (Weakness 2 & Question 3)

  • Format enforcement & prompt template: The Step–SQL Fragment format is specified via prompting and reinforced during training by the sequence-level structure penalty −0.2(1−qfmt)-0.2(1 - q_{\mathrm{fmt}}) in the GRPO reward (Appendix C). We will include the complete training prompt template in the Appendix.
  • Alignment and protocol compliance rates: During training, 99.13%reasoning steps successfully align to unique spans in the final SQL. On BIRD-dev validation, our structure penalty markedly improves protocol compliance over the untrained base model:
Model (BIRD-dev)Full Protocol Compliance (%)Mean Structure Penalty
Qwen3.5-4B (Base)22.83−0.1089
Ours (Trained)97.98−0.0030
  • Inference token and latency overhead: Output token counts and per-query latency on BIRD-dev are summarized below:
MethodBIRD-dev EX (%)Output TokensLatency (s/query)
vanilla OPD61.06539.716.05
GRPO60.011,319.037.07
Ours64.64890.623.91

Compared with vanilla OPD, our method incurs a moderate token increase in exchange for a +3.58-point EX gain, while achieving lower token consumption, lower latency, and higher EX than GRPO—confirming its practical efficiency.

Dear Reviewer egZg,

As the discussion period draws to a close, we wanted to gently check in to see if our responses—including the SERL-SQL integration (+1.37% EX), methodological clarifications, and protocol alignment/latency analyses—have addressed your concerns. We guarantee that all of these new results and clarifications will be thoroughly incorporated into the revised manuscript in accordance with every point of your feedback.

If you have any remaining questions, we would be very glad to continue the discussion; if your concerns have been resolved, we kindly hope you might consider re-evaluating our work. Thank you again for your time and constructive review!

Best regards,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.