acceptodds
Under review as a conference paper at ICLR 2027

RStar-SQL: Self-Evolving Process Preference Learning for Complex-Reasoning Text-to-SQL

Abstract

Complex reasoning Text-to-SQL requires models to jointly resolve schema grounding, implicit constraints, domain knowledge, and SQL construction. While recent MCTS-based methods improve test-time exploration, their search feedback is typically used only for terminal candidate selection, providing limited reusable supervision for subsequent reasoning. We introduce RStar-SQL, a self-evolving process preference framework that converts SQL search outcomes into persistent training signals. RStar-SQL performs schema-grounded MCTS to generate diverse reasoning trajectories and evaluates them using execution consistency together with SQL-specific semantic signals, including schema grounding, execution behavior, join validity, and type operator compatibility. Verified high-quality trajectories refine the policy model, while preferred and dispreferred trajectory pairs train a SQL Process Preference Model. The learned SQL-PPM then guides action selection in subsequent MCTS rounds before terminal feedback is available, forming an iterative loop of search, verification, and policy preference refinement. Experiments on LogicCat and Archer show that RStar-SQL achieves execution accuracies of 35.41% and 47.82%, respectively, demonstrating strong effectiveness on complex reasoning tasks.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Text to SQL, PPM, NLP, Math Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4rjjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RStar-SQL, which is an iterative search-and-training framework for text-to-SQL tasks. It uses MCTS to generate reasoning trajectories and evaluates candidate SQL using execution consistency and checks many SQL specific signals. The selected trajectories provide SFT data, and preferred/dispreferred trajectory pairs train a separate PPM that guides subsequent searches. The paper evaluates 7B and 8B Qwen models on several text-to-sql benchmarks and compares against many out-of-box LLMs, and prior studies.

Critical strengths

Unlike many RL and harness papers in the text-to-sql field, the paper propose an method to integrate SQL verification into iterative search and training and shows empirical gains on text-to-SQL tasks compared to the SFT baseline.

Critical weaknesses
  • Novelty contribution of this paper is limited. The contribution appears to be a SQL-specific supervision and integration recipe built on established MCTS and preference-learning methods. The paper appear to follow established Text-to-SQL search design, such as alpha-sql. In addition, Reasoning-SQL, Arctic-Text2SQL-R1, DualSQL, and ReEx-SQL also have similar design of composite SQL reward shaping. Several of the SQL-specific correctness constraints underlying the proposed signals are precedented in previous studies, such as DIN-SQL and Tool-SQL.

  • The current experiments do not sufficiently establish an advantage over the straightforward SFT + RL recipe with only terminal reward (e.g. Arctic-Text2SQL-R1), especially when the proposed SQL-specific verification signals could also serve as richer multi-step RL rewards.

Questions for authors
  • Can you elaborate more on which specific mechanism goes beyond Alpha-SQL’s structured MCTS? The methodological distinction and experimental controls are not sufficiently clear in this paper.

  • How does RStar-SQL compare with an SFT + RL baseline initialized from the same checkpoint and using the same SQL-specific reward signals? Does its advantage persist under comparable training resources and identical inference/search budgets?

  • It is not clear in the paper, even in the appendix, what the training setups are for the baseline methods. It is hard for the audience to build a mental model how different methods should be fairly compared.

  • From figure 4, I find it difficult to attribute the accuracy improvement entirely to the algorithm because the preference model learns from synthetic trajectory pairs, and subsequent rounds accumulate more training examples and perform additional training. A better comparison might be training the SFT baseline on exactly the same synthetic trajectories, and evaluate it using Alpha-SQL under the same inference budget.

  • Experiments on bird-interact and spider-2 could be some stronger signals to measure its capabilities on complex sql generation tasks.

6 comments
AuthorsOpenReview ↗

Respone to Reviewer 4rjj(1)

Thank you for the comments on methodological distinction, alternative training objectives, and reproducibility. We address the weaknesses and specific questions below.

W1. Novelty relative to existing SQL search and reward designs

We acknowledge the precedents for structured MCTS, SQL constraints, composite rewards, and preference learning. Our contribution focuses on how SQL verification evidence is converted into trajectory preferences and reused to guide subsequent search.

Specifically, execution consistency is augmented with schema and value grounding, execution behavior, join validity, and type–operator compatibility. The resulting reward is propagated through MCTS, and the mean and minimum action values along a trajectory support trajectory selection and preference construction. SQL-PPM then learns to score a reasoning prefix with a candidate action before terminal verification is available.

The component comparison below separates the base-policy and SFT-policy settings and examines verification and preference guidance within those settings. “SQL verification” denotes execution consistency augmented with the four verification signals.

Table R3-1. Component comparison on LogicCat. EX (%).

PolicySearch configurationEX (%)
Base policyMCTS + execution consistency (Alpha-SQL baseline)15.71
Base policyMCTS + SQL verification17.88
SFT policyMCTS + execution consistency25.43
SFT policyMCTS + SQL verification30.46
SFT policyMCTS + SQL verification + SQL-PPM31.17

These results support contributions from verification and learned search guidance. We will clarify the scope of novelty without treating established search operations or individual SQL checks as unique to this work.

W2. Comparison with SFT + RL using the same SQL signals

We include RL controls initialized from the same SFT checkpoint. One uses final SQL execution correctness; the other combines final correctness with the same four SQL verification signals.

Table R3-2. Training-method comparison from the same SFT initialization on LogicCat. EX (%).

MethodTraining feedbackEX (%)
SFT + RLFinal SQL execution correctness22.5
SFT + RL + SQL signalsFinal correctness + the same four verification signals as action-level rewards23.6
RStar-SQLTrajectory supervision and preference-guided search24.6

Adding SQL signals improves the RL control by 1.1 percentage points. RStar-SQL obtains a further 1.0-point gain over this control and a 2.1-point gain over final-reward RL. This provides evidence for our use of the verification information in this experimental setting; we do not interpret it as a general advantage over all RL recipes.

The final-reward control is not a full reproduction of Arctic-Text2SQL-R1. In the SQL-signal control, the verification signals are incorporated as action-level rewards during RL training.

Q1. Mechanisms beyond Alpha-SQL’s structured MCTS

The distinction has three parts:

  1. Inherited search foundation. Structured MCTS and generic SQL constraint checking are established components, not independent novelty claims.
  2. Construction of learnable preferences. SQL verification evidence contributes to search values and trajectory-value aggregation, which are used to construct relative preferences for SQL-PPM.
  3. Reuse during subsequent search. The verifier evaluates generated SQL, whereas SQL-PPM scores the current prefix and candidate action before terminal verification is available. Table R3-1 examines the effects of verification and preference guidance within each policy setting.

We will also sharpen the related-work discussion. Reasoning-SQL uses SQL-tailored partial rewards for GRPO; Arctic-Text2SQL-R1 emphasizes lightweight execution rewards; and ReEx-SQL combines execution-aware training with tree-based decoding. Our distinction concerns preference construction and learned search guidance, rather than a broad claim that existing work lacks execution feedback or tree search.

AuthorsOpenReview ↗

Response to Revewer 4rjj(2)

Q2. Comparable training resources and inference budgets

Thank you for highlighting the importance of resource-controlled comparisons. The RL controls start from the same SFT checkpoint, and the SQL-signal baseline incorporates the same verification signals as action-level rewards, rather than only as a terminal composite reward. RStar-SQL achieves 24.6% EX, compared with 23.6% for this baseline.

We have not collected complete end-to-end resource statistics for this comparison. Our separate inference-cost diagnostics do not provide comprehensive accounting of policy updates, SQL-PPM training, and search-time scoring across these methods. We therefore limit our conclusion to the observed 1.0-percentage-point accuracy gain under the reported initialization and feedback conditions, without claiming superior accuracy or efficiency at equal total compute.

Q3. Baseline training settings and reproducibility

We agree that baseline-specific training details are needed to assess fairness and reproduce the comparisons. Where applicable, our local implementations use common policy initialization, training data, shared optimization settings, and evaluation protocols. Method-specific differences concern the supervision objective, reward design, and search or preference-guidance mechanism.

The main protocol settings recorded in our experiments are summarized below. The filtering controls in Table R2-2 use the separately stated one-epoch, 2,048-token configuration.

Table R3-3. Main policy, search, and preference-model settings.

ModuleHyperparameterSetting
Policy SFTOptimizerAdamW
Policy SFTLearning rate2 × 10⁻⁵
Policy SFTTraining epochs3
Policy SFTMaximum sequence length4,096
Policy SFTPrecisionBF16
Policy SFTPer-device batch size1
Policy SFTGradient accumulation2
Policy SFTEffective global batch size16 on 8 GPUs
MCTSMaximum search depth6
MCTSRollouts8
MCTSUCT coefficient1.4
MCTSRandom seed42
MCTSConcurrent workers64
GenerationReasoning temperature0.2
GenerationSQL generation temperature0.8
GenerationCandidates per SQL generation/revision action5
GenerationSQL execution timeout60 s
SQL-PPMObjectiveBradley–Terry ranking loss
SQL-PPMLearning rate2 × 10⁻⁵
SQL-PPMTraining epochs3
SQL-PPMMaximum sequence length2,048
SQL-PPMPer-device batch size8
SQL-PPMMaximum preference pairs per question2
InferenceEnginevLLM
InferenceMaximum context length8,192
AuthorsOpenReview ↗

Response to Revewer 4rjj(3)

Table R3-4. Reported RL baseline settings.

CategoryHyperparameterSetting
OptimizationRL algorithmGRPO
OptimizationOptimizerAdamW
OptimizationLearning rate1 × 10⁻⁶
OptimizationWeight decay0.01
OptimizationTraining steps100
OptimizationBatch size8 or 16
OptimizationRollout group size8
OptimizationTrajectories per update64 or 128
OptimizationPolicy update epochs1
OptimizationPPO/GRPO clip range0.2
OptimizationKL coefficient0.001
OptimizationGradient clipping1.0
OptimizationPrecisionBF16
OptimizationMaximum sequence length4,096
GenerationSampling temperature0.8
GenerationTop-p1.0
GenerationMaximum response length2,048
RewardFinal correctnessSQL execution correctness
RewardAdditional SQL signalsWeighted semantic verification as action-level rewards
HardwareGPUs8
TrainingFrameworkDeepSpeed ZeRO-3 + vLLM

We will make the baseline-specific configurations, including the per-run batch-size and trajectory-count assignments, explicit in the revised appendix.

Q4. SFT trained on identical synthetic trajectories and evaluated with Alpha-SQL

We include the requested data control: the SFT baseline uses the same training trajectories and positive-filtering rules, followed by Alpha-SQL search.

Table R3-5. Same-trajectory training control on LogicCat. EX (%).

MethodInferenceEX (%)
SFT trained on the same trajectoriesAlpha-SQL search21.5
RStar-SQLFull RStar-SQL search25.3

The 3.8-percentage-point improvement indicates that access to the synthetic trajectories alone does not explain the observed difference. This comparison controls the data factor; it does not by itself control total optimization or inference cost. We will specify the evolution rounds included in the trajectory set and report the associated training and search workload.

Reviewer_YxmjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RStar-SQL, a self-evolving framework which integrates schema-grounded MCTS, SQL-specific semantic signals and process preference model for complex Text-to-SQL. The method generates diverse reasoning trajectories to finetune the policy model and trains a SQL-PPM to guide subsequent searches, achieving performance improvements through iterative updates. Experiments on LogicCat and Archer demonstrate the execution effectiveness of RStar-SQL.

Critical strengths
  1. The method transforms the trajectories generated by SQL searches into reusable training signals, thereby creating a self-evolving cycle between policy learning and search guidance.
  2. The verification mechanism incorporates SQL-specific factors such as schema grounding, join validity, and type compatibility, providing a richer form of supervision than relying solely on execution consistency.
  3. The experiments encompass both complex reasoning tasks and standard Text-to-SQL benchmarks, offering a comprehensive overview of the contributions made by each module.
Critical weaknesses
  1. SQL-PPM is trained on preferences over complete trajectories but used to score partial trajectories and candidate actions during search. It remains unclear whether it learns intermediate decision quality or relies primarily on the final SQL, warranting direct validation of its ability to rank intermediate actions.
  2. Since execution agreement and structural checks may not detect shared semantic errors, could the authors clarify how such errors can be filtered when constructing positive trajectories for subsequent training?
  3. Given the significant time overhead associated with MCTS-based test-time scaling methods, it remains unclear whether the time and resource costs of this framework are acceptable, particularly in practical applications.
Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Revewer Yxmj(1)

Thank you for highlighting the reuse of search trajectories and the role of SQL-specific verification. We address the questions on intermediate action ranking, training-data reliability, and computational cost below.

W1. Whether SQL-PPM can rank intermediate actions

Thank you for suggesting a direct evaluation of intermediate action ranking. At search time, SQL-PPM scores the reasoning prefix together with the candidate action; for schema selection, value linking, and operator identification, the final SQL is not yet available.

MethodPairwise accuracy (%)Top-1 action accuracy (%)Selected-action continuation EX (%)
Random50.061.217.3
SQL-PPM64.778.024.8

In a preliminary evaluation, SQL-PPM achieves 64.7% pairwise accuracy and 78.0% top-1 action accuracy, compared with 50.0% and 61.2% for random selection. Its selected actions yield 24.8% continuation EX versus 17.3%, a gain of 7.5 percentage points.

Ranking metrics are based on four informative groups out of 18, while continuation EX includes all 18 groups. We therefore interpret these results as preliminary evidence.

W2. Filtering shared semantic errors from positive trajectories

Execution consistency and structural checks provide weak evidence, and multiple candidates can share the same semantic error. Our implementation therefore applies reference-execution filtering to individual training trajectories: for training questions with reference SQL, a candidate can enter the positive pool only if its terminal SQL returns the same execution result as the reference query. Eligible trajectories are then ranked using their action values. Consequently, agreement among incorrect candidates is insufficient for positive admission when their result differs from the reference result.

Reference SQL and its execution results are used only for offline training-data filtering. Test-time inference requires neither. Matching results on a single database can still reflect accidental equivalence, so the filter is not a guarantee of full semantic correctness.

Filtering statistics and author audit

We examined terminal trajectories generated for 10,000 Round-0 inputs. Before reference filtering, the pool contains nonempty terminal SQL from the same fixed candidate set. After filtering, candidates must additionally satisfy the recorded reference-execution match. Other selection rules remain unchanged.

We randomly sampled 150 trajectories from each pool using seed 20251113. The authors checked the question and reference SQL for missing conditions, incorrect formulas or aggregations, join errors, and value-grounding errors. This is an author audit, not an independent annotation study. Because post-filter execution matching is true by construction, we do not use its 100% match rate as independent evidence of quality.

Table R2-1. Reference-execution filtering and author-audit results.

Candidate poolTrajectoriesRetention (%)AuditedCases labeled as semantic errorsCases labeled as uncertain
Before reference filtering43,459100.015037 (24.7%)32
After reference filtering25,84759.51500 (0.0%)2

The retained pool contains 25,847/43,459 trajectories (59.5%). Among the 7,283 questions represented in the candidate pool, 5,649 retain at least one matching trajectory. Reference SQL text is available for all questions; 58 empty-SQL trajectories are excluded from both pools. These statistics reuse the MySQL execution-match records from trajectory construction; reference-execution failures were not separately rerun in this audit.

Before filtering, the 37 cases labeled as errors mainly involve incorrect formulas, omitted conditions, wrong tables, or incorrect values. The 32 uncertain cases include queries close to the reference but with execution mismatches involving literals, multilingual values, or aliases. The two retained cases labeled as uncertain raise concerns about accidental equivalence despite omitted constraints: a recent-year condition in one case and a join enforcing an all-five-star condition in the other. We retain the audit’s original labels here and do not interpret the zero confirmed-error count as evidence that residual semantic errors have been eliminated.

AuthorsOpenReview ↗

Response to Revewer Yxmj(2)

Downstream SFT control

We also compare three SFT datasets using the same Qwen3-8B initialization, one training epoch, learning rate 2 × 10⁻⁵, maximum sequence length 2,048, at most two trajectories per question, and the same holdout. Evaluation uses greedy decoding on the 500-question LogicCat development set. The filtered condition reuses the existing checkpoint trained on the same 9,866 reference-filtered trajectories.

Table R2-2. Effect of reference filtering on downstream SFT.

Training dataTrajectoriesCorrect / 500EX (%)
Unfiltered, all available training trajectories13,13295/50019.0
Unfiltered, randomly size-matched9,866102/50020.4
Reference-execution filtered9,866121/50024.2

The filtered condition improves over the size-matched unfiltered condition by 3.8 percentage points. This supports the usefulness of reference filtering beyond simply changing the number of training trajectories, while leaving accidental execution equivalence as a limitation.

W3. Inference time and resource cost

We report both accuracy and cost using the same base 8B policy, a maximum search depth of six, and eight rollouts. This comparison isolates the inference components; it does not use the fully evolved policy from the main results. Single-pass generation uses greedy decoding (temperature zero), includes the schema, and makes one generation call. The Alpha-SQL baseline uses execution consistency, while SQL verification adds the four verification signals.

Table R2-3. Accuracy and per-query inference cost on 500 LogicCat questions.

MethodEX (%) [correct / 500]Mean latency (s/query)Policy-generated tokens/queryPPM input tokens/query
Single-pass generation9.2 [46/500]1.091,1690
MCTS + execution consistency (Alpha-SQL baseline)15.8 [79/500]216.771,1510
MCTS + SQL verification18.2 [91/500]222.771,2820
MCTS + SQL verification + SQL-PPM20.4 [102/500]234.6167,63134,239

In this setting, verification and preference guidance improve EX from 15.8% to 20.4%, with latency increasing from 216.7 to 234.6 seconds per question. Token consumption also increases substantially, so the latency difference should not be interpreted as an equivalent increase in total computation. The multi-minute query cost remains a limitation for latency-sensitive applications.

We additionally vary the rollout budget while holding the base 8B policy and maximum depth fixed.

Table R2-4. Accuracy–latency trade-off under different rollout budgets.

RolloutsAlpha-SQL EX (%) [correct / 500]Alpha-SQL latency (s/query)Verification + PPM EX (%) [correct / 500]Verification + PPM latency (s/query)
212.2 [61/500]71.514.6 [73/500]82.3
415.2 [76/500]124.619.4 [97/500]141.1
815.8 [79/500]216.720.4 [102/500]234.6

The four-rollout guided configuration obtains 97 correct answers in 141.1 seconds per query, compared with 79 correct answers and 216.7 seconds for the eight-rollout consistency baseline. This illustrates a useful accuracy–latency trade-off in the reported configuration. It does not establish a universal cost advantage, and equal rollout counts should not be treated as equal total compute.

Finally, we sincerely thank the reviewer for the constructive feedback. We have incorporated some of the additional material and wording revisions into the revised manuscript.

Reviewer_YxmjOpenReview ↗

Thanks for your response.

Thanks for your overall response that addresses most of my concerns. For the additional evaluation of intermediate action ranking, could you give a concrete example of an evaluation group and explain what made it one of the four “informative” groups out of 18, and how you determined which candidate action was best? Please also clarify whether the same group selection and labeling criteria were used for the top-1 action-ranking results in Figure 5(b).

Reviewer_1MqQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RStar-SQL, a self-evolving process preference framework for complex-reasoning text-to-SQL. Building on schema-grounded Monte Carlo Tree Search (MCTS), RStar-SQL generates multiple reasoning trajectories and evaluates them using execution consistency and SQL-specific semantic signals. It then uses these trajectories to refine the policy model and train process reward models (PRMs). The paper evaluates RStar-SQL on LogicCat and Archer, comparing its performance with existing text-to-SQL methods.

Critical strengths

S1. The paper studies an important problem of improving complex reasoning in text-to-SQL through search and process preference learning.

S2. RStar-SQL provides a reasonable framework that combines schema-grounded MCTS with policy refinement and PRM training, supported by experimental evaluation on challenging text-to-SQL benchmarks.

Critical weaknesses

W1. The technical novelty of RStar-SQL is limited. Its schema-grounded MCTS for generating and evaluating reasoning trajectories is similar to Alpha-SQL, while the subsequent process preference learning and PRM training closely follow SQL-PPM. The paper appears to mainly combine these existing techniques, without clearly introducing new mechanisms beyond their integration. The authors should better clarify the differences from Alpha-SQL and SQL-PPM.

W2. The comparison with Alpha-SQL is not entirely fair. Alpha-SQL is a training-free method, while RStar-SQL further trains the policy model and PRMs using generated reasoning trajectories. The performance gains may therefore come from additional training rather than the proposed search and preference mechanisms. The paper should provide a controlled comparison under comparable training settings to better demonstrate the contributions of RStar-SQL.

Questions for authors

Q1. What are the main technical contributions of RStar-SQL beyond combining Alpha-SQL and SQL-PPM?

1 comment
AuthorsOpenReview ↗

Response to Revewer 1MqQ

Thank you for recognizing the importance of complex-reasoning Text-to-SQL and for the suggestions on methodological positioning and experimental controls.

W1. Technical contribution and distinction from prior work

RStar-SQL builds on established MCTS and process preference learning. Its contribution centers on how SQL-specific verification supports trajectory evaluation, preference construction, and subsequent search guidance (Sections 3.2–3.3).

SQL-specific verification. Execution agreement alone is insufficient: multiple incorrect queries can return the same result. We augment execution consistency with four complementary signals covering schema and value grounding, execution behavior, join validity, and type–operator compatibility. These signals address different SQL failure modes and provide weak ranking evidence rather than guarantees of semantic correctness.

From verification to process preferences. The verification reward is propagated through standard MCTS backpropagation. We then use the mean and minimum action values along a trajectory to support positive-trajectory selection and preference-pair construction, including high-valued unsuccessful trajectories as hard negatives. SQL-PPM learns from these preferences and subsequently scores candidate actions together with their reasoning prefixes, before terminal verification is available. The contribution lies in this use of SQL verification evidence to construct reusable process supervision.

Empirical support. In manuscript Table 2, adding all semantic verification signals improves EX from 26.34% to 28.32% on LogicCat and from 43.26% to 44.67% on Archer, gains of 1.98 and 1.41 percentage points, respectively. Removing individual signals also reduces performance. The following component comparison further separates policy training, verification, and preference guidance. Here, “SQL verification” denotes execution consistency augmented with the four verification signals.

Table R1-1. Component comparison on LogicCat. EX (%).

PolicySearch configurationEX (%)
Base policyMCTS + execution consistency (Alpha-SQL baseline)15.71
Base policyMCTS + SQL verification17.88
SFT policyMCTS + execution consistency25.43
SFT policyMCTS + SQL verification30.46
SFT policyMCTS + SQL verification + SQL-PPM31.17

We acknowledge that SQL checks, composite rewards, preference learning, and policy–preference evolution have precedents. IESR also uses execution and semantic consistency evidence to rank search candidates. We will clarify these connections and distinguish the inherited components from our SQL-specific supervision design, rather than presenting MCTS or generic constraint checking as new contributions.

W2. Fairness of the comparison with Alpha-SQL

We agree that comparing a trained system with a training-free search method does not isolate the source of improvement. We therefore include a control in which an SFT baseline receives the same synthetic training trajectories and positive-filtering rules and is evaluated using Alpha-SQL search.

Table R1-2. Same-trajectory training control on LogicCat. EX (%).

MethodTraining trajectoriesInferenceEX (%)
SFT + Alpha-SQLSame synthetic trajectoriesAlpha-SQL search21.5
RStar-SQLSame synthetic trajectoriesFull RStar-SQL search25.3

RStar-SQL improves by 3.8 percentage points in this comparison. This supports an additional benefit beyond access to the synthetic trajectories alone. The experiment controls the training-data factor; it does not by itself establish equivalence in total training or inference compute. We will specify the trajectory rounds, optimization workload, and search costs in the revised experimental description. Finally, we sincerely thank the reviewer for the constructive feedback. We have incorporated some of the additional material and wording revisions into the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Manuscript Revision

We thank the reviewers for their constructive feedback. We have conducted additional experiments and revised the manuscript to address the main concerns. Major manuscript revisions are highlighted in blue.

1. Reviewer 1MqQ: methodological distinction and comparison fairness

  • Contribution clarification. We revised the Introduction, Methodology, and Related Work to distinguish inherited MCTS and preference-learning components from our construction and reuse of SQL-specific process supervision (Sections 1, 3.2–3.3, and 5).
  • Controlled comparisons. We added component controls separating policy training, SQL verification, and preference guidance, and reported a same-trajectory SFT + Alpha-SQL comparison to examine whether improvements arise solely from additional training data (Appendix B.1; rebuttal Tables R1-2 and R3-5).

2. Reviewer Yxmj: intermediate guidance, supervision reliability, and inference cost

  • Intermediate-action evaluation. We added the evaluation protocol, labeling and aggregation rules, and a concrete example explaining informative groups and best-action selection. We also clarified their relationship to Figure 5(b). The reviewer confirmed that the follow-up addressed their concern (Appendix B.5).
  • Training-data reliability. We clarified offline reference-execution filtering and added filtering statistics, author audits, and size-matched SFT controls, while documenting the limitation of accidental execution equivalence (Section 4.3, Table 4).
  • Computational cost. We added fixed-policy inference-cost comparisons and rollout-budget experiments, reporting latency and token workloads separately and clarifying practical limitations (Appendix B.2).

3. Reviewer 4rjj: RL alternatives, reproducibility, and broader evaluation

  • RL controls. We reported same-initialization RL comparisons using terminal correctness and the same SQL verification signals as action-level rewards, while explicitly limiting claims about equal-compute comparisons (rebuttal Table R3-2).
  • Reproducibility. We expanded baseline-specific training, reward, search, and inference settings (Appendices B.4 and C.1).
  • Broader SQL evaluation. We added experiments on the available Spider 2.0-Lite SQLite subset and BIRD-INTERACT under a static, single-turn protocol, with explicit coverage and protocol limitations (Appendix B.3).

We appreciate the reviewers’ time and hope these clarifications, additional experiments, and manuscript revisions assist the Revewer’s assessment. Overall, we believe the above revisions strengthen the empirical evidence for the effectiveness of our work. We would be happy to continue the discussion if there are any further questions or comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.