- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The proposed method try to combine the benefit of RLVR and OPD by re-weighting on SQL part generation. The re-weighting includes three steps 1/ assigns predefined shares of the total distillation weight to final SQL and alignable reasoning steps 2/ it uses ASTs to parse the SQL structure 3/ Step–SQL Fragment protocol pairs are extrated to align reasoning step with SQL generation. From the result, it shows the proposed method outperforms OPD in BIRD and Spider benchmark from 1.75 to 3.58 in EX.
Critical strengths
1/ The idea of reweighting the SQL part and aligned the reasoning part better is reasonable. 2/ The performance gain compared to OPD makes sense.
Critical weaknesses
The experiment is not comprehensive: 1/ No teacher model perrformance. 2/ Only one student model is tested. 3/ No RLVR performance. 4/ the ablation of Table 2, only test BIRD. It should covered Spider as well.
Minor: 1/ the illustrating examples in Figure 1 and 2 are different and redundant 2/ line 338, the Claude Sonnet-3.7 should be model instead of Method
Questions for authors
N/a
3 comments
Dear Reviewer xChW,
As the discussion period comes to an end soon, we wanted to gently follow up to see if our responses have addressed your concerns.
Specifically, we clarified the misunderstanding regarding Points 1 and 2 (pointing to the 27B teacher and additional 9B student results in Section 4.8 and Appendix E, which we will move directly into Table 1 for immediate visibility), added the controlled RLVR (GRPO-only) experiments, and conducted the full A0–A4 ablation study on Spider. We guarantee that we will carefully incorporate every point of your feedback—including these experiments and the figure/table fixes—point by point into our revised manuscript.
We would be glad to discuss any remaining questions before the deadline, and if your concerns are resolved, we kindly hope you will reconsider our work. Thank you again for your valuable feedback!
Best regards,
Thanks for the pointing the experiments result in Section 4.8 and Appendix E. I would keep my current score of 2 (from 1.8 -> 2.4).
We thank the reviewer for the constructive comments and suggestions. Below, we address each concern point by point.
1. Teacher Model Performance and Additional Student Model Size
We actually evaluated and reported both the Qwen3.8-27B teacher model and an additional Qwen3.5-9B student model in Section 4.8 (Effectiveness Across Student Model Sizes) and Appendix E (Figure 4) of the manuscript:
In the revised manuscript, we will also include the 27B teacher and the 9B student results directly in the main results table (Table 1) to make these comparisons immediately visible.
2. RLVR (GRPO-Only) Performance
Following your suggestion, we added a controlled GRPO-only baseline, while keeping the exact same Qwen3.5-4B student, training data, and task reward setup). As shown in the table below:
3. Ablation Study on Spider
We conducted an additional ablation study trained on the Spider training split and evaluated on both Spider-dev and Spider-test across configurations A0–A4. As shown in the table below, our method remains consistently effective on Spider:
Note: ✓/-- denotes enabled/disabled. A0 is vanilla OPD; A4 is the complete method. Δ denotes the change relative to A0 in percentage points.
4. Minor Issues
Thank you for pointing out these presentation details:
Claude Sonnet-3.7in Table 1 and carefully proofread all tables and notations throughout the paper.