- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper fixes Text-to-SQL translator and optimizes database schema via evolutionary search. It significantly improves the execution accuracy in 13 DBs and 101 questions. But the improvement in held-out is not that much.
Critical strengths
This paper has practical known-workload optimization value. After fixing the translator, it achieves obvious gain and reduces "join" in generated SQL. It provides empirical evidence for reducing task difficulty via derived tables.
Critical weaknesses
-
It's not very clear about the independent contribution. I think the key gain is mainly from training questions, and lacking the comparison with documentation-only and direct view generation under same supervision and budget. Thus the value of evolutionary search is uncertain.
-
Row-count check can not guarantee full preservation of source rows/values. Thus, I think it can not support the claimed hard constraint.
Questions for authors
Could you please tell me more about the training/held-out execution accuracy on documentation-only and direct view generation under same training feedback and budget?
Thank you for the review and for stating clearly what was missing.
W1 and Question. Independent contribution; documentation-only and direct view generation under the same supervision and budget
To separate these contributions we added an experiment that was not in the submitted version (Section 5.4, Table 2). In it, the schema is written without any question, by the two simpler methods you name. Documentation only (value-format notes that code writes automatically) gives +4.8 points. Direct view generation (one mutator step that sees the database but no question and gets no training feedback) gives +7.5 points. Both together give +10.6 points (95% interval +4.0 to +17.5), for Qwen3.8-27B on 96 benchmark questions.
The evolution with training feedback gives more with the same translator: +26.4 points on the training questions and +14.9 points on 194 unseen questions, which are benchmark questions with one parameter changed and which no mutator saw. So the gain is not limited to the training questions.
Revised paper: Section 5.4; Table 1; Table 2.
W2. Row-count check
We agree, and Reviewer FuYR raised the same point (Q2). We replaced the row count with a full comparison of every source table with the original, in schema and in rows as a multiset in both directions, with a fitness of zero on any difference (Section 3.3). Our answer to Reviewer FuYR, Q2, lists the checks in detail. 39 of the 40 delivered schemas pass. The exception is a database whose own schema declares a foreign key that our rules do not let a candidate declare; its rows are identical (Section 5.3).
Revised paper: Section 3.3; Section 5.3, last paragraph.
We would be glad to answer further questions.