- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SQuAR, a schema retrieval method that identifies the target database and the tables required to answer a question in an open pool of multiple databases. In the offline stage, SQuAR samples multi-table sub-schemas along foreign-key paths, generates synthetic questions for each sub-schema, and indexes them as retrieval anchors. At query time, it retrieves anchors similar to the user question and aggregates their associated databases and tables through voting, without any LLM calls. The authors also argue that table recall alone cannot tell whether the retrieved schema is sufficient to construct the target SQL, and they propose two metrics, ERC and ERJ. Across four benchmarks, SQuAR improves Table Recall and ERC over the baselines.
Critical strengths
- The paper is clearly written and easy to follow.
- The motivation is convincing: missing even a single required table makes the correct SQL impossible to generate.
- The experiments are well organized around explicit research questions.
Critical weaknesses
-
The core idea of SQuAR is to generate synthetic questions that cover diverse table combinations and to use them as retrieval anchors so that all required tables are recovered. However, the offline pipeline closely follows DBCopilot [1]. DBCopilot already (i) constructs a schema graph in which a virtual root node connects to database nodes, which in turn connect to their tables linked by key relationships, (ii) samples sub-schemas via random walks starting from the root node, and (iii) generates a pseudo-question for each sampled sub-schema in a schema-to-question manner. SQuAR reproduces these three steps. The main methodological difference lies in how the generated questions are used: DBCopilot trains a generative router on them, whereas SQuAR indexes them for kNN retrieval followed by frequency voting. Moreover, the base models differ substantially. DBCopilot uses T5-base for both its schema-to-question generator and its router, whereas SQuAR uses Qwen2.5-14B-Instruct for question generation and Qwen3-Embedding-4B for retrieval. It is unclear whether the gains over DBCopilot stem from the proposed anchor-based retrieval or simply from stronger base models.
-
Using generated questions to bridge the gap between user queries and retrieval targets is well established in information retrieval. doc2query [2] expands documents with predicted queries, and PAQ [3] goes further: its retriever, RePAQ, directly retrieves the most similar generated question and returns the attached answer. SQuAR’s online stage closely mirrors this paradigm, with table sets in place of answers. In text-to-SQL, the Contextual RAG component of CSR-RAG [4] likewise retrieves the top-k most similar questions from a collection of question–SQL pairs and returns the union of their associated table sets. None of these works are cited or discussed. Furthermore, frequency-based voting over the retrieved anchors is hard to regard as a separate technical contribution.
-
The paper states that an execution-ready schema should contain all required tables while excluding unnecessary ones (Challenge 2). However, SQuAR returns the top-K tables by vote count for a fixed, pre-set K. Apart from removing tables outside the predicted database, it has no query-adaptive mechanism for deciding how many tables to return. Indeed, SQuAR’s ERJ is lower than that of CORE-T [5] on both Spider and BIRD at every K.
[1] Wang et al., DBCopilot: Natural Language Querying over Massive Databases via Schema Routing, EDBT, 2025.
[2] Nogueira et al., Document Expansion by Query Prediction, arXiv, 2019.
[3] Lewis et al., PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them, TACL, 2021.
[4] Singh et al., CSR-RAG: An Efficient Retrieval System for Text-to-SQL on the Enterprise Scale, arXiv, 2026.
[5] Soliman et al., CORE-T: COherent REtrieval of Tables for Text-to-SQL, arXiv, 2026.
Questions for authors
-
Have you compared SQuAR with DBCopilot when both use the same synthetic questions? For example, DBCopilot's router could be trained on the 10K questions generated by Qwen2.5-14B-Instruct, or SQuAR's index could be built from questions produced by DBCopilot's question generator. Such a comparison would clarify whether the gains come from the anchor-based retrieval or from the stronger base models.
-
How does SQuAR compare with a simple baseline that uses the same embedder (Qwen3-Embedding-4B)? One such baseline would densely retrieve tables using their descriptions and columns, and optionally expand the result to neighboring tables via foreign keys. What is the net gain attributable to the anchor-based approach itself?
-
Are there cases where ERC leads to a substantially different conclusion from the existing complete Recall metric? If not, what justifies proposing ERC as a new metric?
6 comments
Response to Critical Weakness W3 and Question Q1
Response to Critical Weakness W3
We acknowledge that SQuAR uses a fixed retrieval budget rather than query-adaptive selection. Our primary objective is complete schema recovery, since missing a required table or selecting the wrong database prevents the construction of the gold SQL.
This trade-off highlights the importance of both ERC and ERJ. ERC measures complete, correctly routed retrieval, while ERJ penalizes unnecessary tables. CORE-T achieves higher ERJ on Spider and BIRD, reflecting better selectivity, whereas SQuAR achieves higher ERC, reflecting better completeness.
Importantly, CORE-T's higher selectivity did not translate into higher downstream execution accuracy (EA) with our fixed GPT-5.5 generator. SQuAR achieves the highest EA among evaluated retrieval methods (Figure 4). However, ERJ may be more valuable for smaller-context models or schemas with extensive per-table metadata, where over-retrieval is more costly. Query-adaptive selection for SQuAR is a promising future extension to improve its selectivity while preserving completeness.
Response to Question Q1
We thank the reviewer for this suggestion. To address this concern, we trained DBCopilot's T5-base router (220M parameters) using the exact same 10K Qwen-generated question–schema pairs used to construct SQuAR's retrieval index. We additionally evaluated SQuAR using UAE-Large-V1 (335M parameters) instead of Qwen3-Embedding-4B (4B) to reduce the disparity in backbone size.
Comparison with DBCopilot using matched synthetic data (ERC@5, %).
| Dataset | DBCopilot (Original) | DBCopilot (Matched 10K) | SQuAR-UAE |
|---|---|---|---|
| Spider | 76.21 | 50.29 | 95.55 |
| BIRD | 68.38 | 35.20 | 89.50 |
| BEAVER-DW | 8.26 | 0.83 | 21.49 |
| BEAVER-NW | 12.50 | 0.00 | 20.45 |
As shown in the table, SQuAR-UAE achieves higher ERC@5 than both the original DBCopilot and its matched-data variant across all four benchmarks. On Spider/BIRD, SQuAR-UAE achieves 95.55%/89.50% ERC@5, compared with 50.29%/35.20% for DBCopilot trained on the identical synthetic dataset.
By using the exact same synthetic questions for both methods and a substantially smaller embedding backbone for SQuAR, these results demonstrate that SQuAR's improvements cannot be attributed solely to the stronger Qwen2.5-14B-Instruct model used for offline question generation or the original Qwen3-Embedding-4B retrieval backbone.
Response to Question Q2 and Question Q3
Response to Question Q2
We thank the reviewer for this suggestion. To directly address this question, we implemented a direct table retrieval baseline using the same Qwen3-Embedding-4B backbone as SQuAR. Tables are represented by their database and table names, columns, key relationships, and purpose descriptions. The baseline performs dense retrieval with one-hop foreign-key expansion.
SQuAR versus Simple Baseline using Qwen3-Embedding-4B (ERC, %).
| Dataset | K | Direct-Table + FK | SQuAR |
|---|---|---|---|
| Spider | 5 | 89.07 | 96.13 |
| 10 | 89.46 | 96.81 | |
| 15 | 89.46 | 96.81 | |
| BIRD | 5 | 83.77 | 91.40 |
| 10 | 92.37 | 97.46 | |
| 15 | 94.52 | 97.52 | |
| BEAVER-DW | 5 | 14.05 | 22.31 |
| 10 | 28.93 | 49.59 | |
| 15 | 43.80 | 66.12 | |
| BEAVER-NW | 5 | 7.95 | 17.05 |
| 10 | 14.77 | 28.41 | |
| 15 | 20.45 | 43.18 |
SQuAR achieves higher ERC in all 12 comparisons, with improvements ranging from 3.00 to 22.73 percentage points. Notably, on BEAVER-DW and BEAVER-NW, SQuAR improves ERC@15 from 43.80%/20.45% to 66.12%/43.18%, respectively. Our existing QGpT and CORE-T comparisons provide complementary evidence. QGpT retrieves tables using enriched representations containing schema metadata, sample rows, and synthetic questions, while CORE-T combines table-level retrieval with LLM-based refinement and joinability information. Using their respective embedding backbones, BGE-M3 (QGpT) and UAE-Large-V1 (CORE-T), SQuAR-BGE achieves 94.78% ERC@5 versus 85.01% for QGpT, while SQuAR-UAE achieves 95.55% versus 87.52% for CORE-T on Spider.
SQuAR versus competing methods using matched embedding backbones (ERC, %).*
| Dataset | K | SQuAR-BGE | QGpT | SQuAR-UAE | CORE-T |
|---|---|---|---|---|---|
| Spider | 5 | 94.78 | 85.01 | 95.55 | 87.52 |
| Spider | 10 | 94.78 | 75.73 | 96.13 | 88.10 |
| Spider | 15 | 94.97 | 75.63 | 95.84 | 87.72 |
| BIRD | 5 | 88.59 | 82.01 | 89.50 | 79.34 |
| BIRD | 10 | 96.28 | 86.96 | 97.13 | 84.8 |
| BIRD | 15 | 96.35 | 86.90 | 97.20 | 85.66 |
| BEAVER-DW | 5 | 16.53 | 15.70 | 21.49 | 10.74 |
| BEAVER-DW | 10 | 39.67 | 27.27 | 42.15 | 16.53 |
| BEAVER-DW | 15 | 55.37 | 34.71 | 60.33 | 16.53 |
| BEAVER-NW | 5 | 18.18 | 2.27 | 20.45 | 9.09 |
| BEAVER-NW | 10 | 29.55 | 5.68 | 32.95 | 20.45 |
| BEAVER-NW | 15 | 53.41 | 7.95 | 48.86 | 17.0 |
SQuAR-BGE outperforms QGpT in all 12 settings, while SQuAR-UAE outperforms CORE-T in all 12 settings. Overall, SQuAR achieves higher ERC in all 24 matched-backbone comparisons, providing further evidence that its gains cannot be attributed solely to its stronger embedding backbone.
Response to Question Q3
We thank the reviewer for this question. ERC can lead to different conclusions about retrieval effectiveness that better align with downstream SQL generation. Unlike Complete Recall (CR), ERC requires both correct database identification and retrieval of all required tables, two essential requirements for generating the target SQL.
Consider IterJAR and CORE-T at K = 15:
Complete Recall (CR), ERC, and downstream execution accuracy (EA) at K = 15 (%).
| Dataset | Method | CR | ERC | EA |
|---|---|---|---|---|
| Spider | IterJAR | 97.49 | 60.44 | 50.0 |
| CORE-T | 95.70 | 87.72 | 71.7 | |
| BIRD | IterJAR | 96.54 | 70.93 | 35.5 |
| CORE-T | 87.16 | 85.66 | 42.0 |
Despite achieving higher CR on both datasets, IterJAR yields lower downstream execution accuracy. On Spider, CORE-T achieves 21.7 percentage points higher EA, despite its lower CR, alongside a 27.3-point improvement in ERC. The same pattern holds on BIRD, where CORE-T achieves 6.5 points higher EA and 14.7 points higher ERC.
These examples illustrate that ERC measures whether retrieval produces an end-to-end-ready schema for downstream SQL generation. As demonstrated in Section 5.5 (Figure 4), higher ERC is generally associated with higher execution accuracy (EA) using the same GPT-5.5 generator. Thus, ERC provides a more task-relevant measure of open-pool schema retrieval by assessing whether the retrieved schema contains the correct database and all necessary tables to support SQL generation.
Response to Critical Weakness W1
Response to Critical Weakness W1
We acknowledge that DBCopilot introduced schema-graph construction, random-walk sampling, and synthetic question generation. Building on these components, SQuAR introduces a different retrieval formulation for open-pool schema retrieval: rather than training a generative router, it directly indexes synthetic question–schema pairs and aggregates retrieved evidence to identify the correct database and recover complete sets of related tables, without autoregressive decoding.
This distinction offers four practical advantages:
(1) Schema extensibility: DBCopilot learns schema routing through a trained generative model, whereas SQuAR can incorporate new databases and tables by generating and indexing additional anchors, without retraining a schema-specific router.
(2) Online efficiency: SQuAR replaces autoregressive schema decoding with dense retrieval and evidence aggregation, achieving 3.5–11.2× faster median retrieval than DBCopilot across our four benchmarks (Table 7).
(3) Synthetic-example efficiency: DBCopilot originally uses 100K synthetic training examples per collection, whereas SQuAR uses only 10K anchors per benchmark. With 10× fewer synthetic examples, SQuAR-UAE achieves 19.34 and 21.12 percentage points higher ERC@5 on Spider and BIRD, respectively, compared with the original DBCopilot.
(4) Order-independent evidence aggregation: Rather than generating schema elements as an autoregressive sequence, SQuAR directly aggregates database and table evidence from multiple retrieved question–schema pairs, without relying on an ordering of schema elements during decoding.
To address backbone fairness, we additionally evaluated SQuAR using UAE-Large-V1 (335M) instead of Qwen3-Embedding-4B. Since DBCopilot employs T5-base (220M) as a generative router rather than a dedicated embedding model, UAE-Large-V1 provides a comparison at a similar parameter scale, although the architectures differ. SQuAR-UAE achieves 95.55%/89.50% ERC@5 on Spider/BIRD, compared with 76.21%/68.38% for DBCopilot. These results show that SQuAR's gains do not depend solely on its original 4B-parameter embedding backbone.
To control for differences in synthetic question generation, we additionally trained DBCopilot's T5-base router using the exact same 10K LLM-generated question–schema pairs used to construct SQuAR's retrieval index. Under this matched-data setting, DBCopilot achieves 50.29%/35.20% ERC@5 on Spider/BIRD, compared with 95.55%/89.50% for SQuAR-UAE. Although the retrieval architectures remain different, these results provide complementary evidence that SQuAR's improvements are not solely attributable to differences in synthetic question generation or its original larger embedding backbone.