acceptodds
Under review as a conference paper at ICLR 2027

Within and Beyond SQL: Searching for Execution Evidence in Text-to-SQL Verification

Abstract

Text-to-SQL maps natural language questions to executable SQL queries over databases. Reliable text-to-SQL requires verifying whether generated SQL faithfully satisfies the semantics of the question. Existing verification remains challenging because intermediate discrepancies may be masked by downstream operations, while omitted requirements leave no corresponding computation from which execution evidence can be directly obtained. We introduce EviSQL, a reference-free verification framework that searches for execution evidence both within and beyond the computation expressed by candidate SQL. Within the SQL, EviSQL reconstructs the computation trace of SQL and tests selected operations through local perturbations, tracking whether the resulting differences propagate to the final answer. Beyond the SQL, EviSQL grounds question requirements in the original wording and relates them to the computation trace, using intent-bound queries to investigate possible semantic coverage gaps. Experiments on \rose and \bird show that EviSQL consistently improves semantic error detection over existing verification methods. The resulting execution evidence further supports fine-grained error diagnosis and targeted SQL correction, improving the reliability of generated queries. Code and experimental resources are available at https://anonymous.4open.science/r/EviSQL-30D4.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.