- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents EviLink, which at its core resembles a two-step pipeline for text-to-SQL that intends to consider multiple SQL candidates for aligning with the underlying schema until committing to one. The first step generates multiple SQL candidates and informs which SQL / schema elements are necessary and which are uncertain, after which the second investigates the uncertain elements further in an agentic manner. The evaluation shows that EviLink holds some promise, but comes with several limitations.
Critical strengths
S1 The idea to consider multiple SQL paths before committing to one SQL, as done in EviLink, is promising and the motivation aligns with earlier findings.
S2 Using candidate overlap to inform common hence more certain schema elements versus uncertain ones that guide the agentic data exploration seems an efficient approach.
S3 The evaluation includes most prior relevant approaches as baselines, relevant metrics and datasets, although the related DisambigParse [1] is missing.
Critical weaknesses
W1 Methodological weaknesses:
- W1.1 SQL path hypothesis generation isn’t developed in a principled manner, while it is one of the core ideas of the paper. In its current form, SQL hypothesis generation relies fully on the LLM’s capability to generate plausible and diverse SQL queries. Moreover, K is set “adaptively” without explanation, whereas K can also be a single hypothesis.
- W1.2 The uncertainty assessment is based on an LLM’s own assessment through a dense prompt given the collected evidence elements, rather than a reliable measure.
W2 The performance analysis shows that while EviLink is effective on Spider-Snow, EviLink still has significant shortcomings as its SRR is relatively far off from the best-performing approach and its precision is compromised significantly. On Spider-Snow EviLink indeed performs better but it isn’t shown that the database sizes in Spider-Snow explains this.
W3 The ablation study confirms that while multiple SQL path hypotheses are effective for improving recall, it significantly sinks the precision (hence F1). Meanwhile, the uncertainty-grounded refinement isn’t contributing as much to the final performance.
W4 Altogether, the contribution made in this paper is not solid, novel nor substantial enough for publication in its current form.
[1] Saparina, I., & Lapata, M. Disambiguate first, parse later: Generating interpretations for ambiguity resolution in semantic parsing. In Findings of the Association for Computational Linguistics: ACL 2025.
Questions for authors
Q1 How is K set adaptively in multi-hypothesis SQL generation?
Q2 It would be interesting to explain the evidence-guided agentic exploration with a concrete example.
2 comments
Rebuttal - Part II
4. A concrete evidence-guided refinement example
Q2 It would be interesting to explain the evidence-guided agentic exploration with a concrete example.
Figure 2 illustrates the grounding-to-refinement pipeline on the real Spider2-Snow example sf_bq391. Below, we provide a turn-by-turn trace of another example, sf_local163, to show how the agent uses evidence and makes decisions.
In sf_local163, refinement reduces 11 fields to 6 while retaining all four reference fields. The question asks for faculty salaries closest to their rank's mean, returning rank, first name, last name, and salary. Refinement starts with L2 evidence, including available field descriptions, sample values, and null ratios, to assess field relevance. The current GLM trace records:
| Step | Recorded action | Effect |
|---|---|---|
| Grounding | Three paths select one table and 11 fields | Keeps four reference fields plus seven extras |
| Turn 1 | Five successful discard calls remove FacState, FacCity, FacDept, FacHireDate, and FacSupervisor | Prunes attributes using default L2 evidence and query scope |
| Turn 2 | list_columns checks STAFFHOURS, SALARYDATASET, and UNIVERSITY_STUDENT | Inspects candidates; adds no fields |
| Turn 3 | keep receives an empty item list | Returns six fields with all four reference fields retained |
| Field metric | Before refinement | After refinement |
|---|---|---|
| Selected fields | 11 | 6 |
| Reference fields retained | 4/4 | 4/4 |
| Precision | 36.36% | 66.67% |
| Recall | 100.00% | 100.00% |
| F1 | 53.33% | 80.00% |
This trace shows how evidence-guided pruning improves precision while retaining recall in this example; two extra fields remain. It does not imply universal gains. We will add this walkthrough in the revised manuscript.
5. Relation to DisambigParse
S3 The evaluation includes most prior relevant approaches as baselines, relevant metrics and datasets, although the related DisambigParse [1] is missing.
We thank you for highlighting DisambigParse. Our introduction already cites the AmbiQT and AMBROSIA papers (Bhaskar et al., 2023; Saparina and Lapata, 2024) to motivate preserving multiple valid SQL implementations. DisambigParse addresses this related problem by generating natural-language interpretations, filling in missing meanings, and parsing each into SQL. EviLink focuses on schema linking: grounding alternative SQL paths separately and using their schema requirements to guide refinement. We will add a comparison with DisambigParse in the related work of the revised manuscript.
Rebuttal - Part I
We sincerely thank you for recognizing multi-path preservation, the potential efficiency of overlap-guided exploration, and the evaluation choices. Your questions help clarify hypothesis generation and its benefits. Below, we address them through methodological explanations, experimental results, and a concrete example.
1. How the number of hypotheses is chosen
In §3.1, K is the fixed maximum budget (default 4), while M ≤ K is the actual number of hypotheses retained. The prompt encourages coverage of distinct join routes, filters, aggregation, or temporal interpretations within this budget. Hypotheses describe plausible SQL realizations and need not be executable queries. Each drives its own table and field selection; we preserve these grounded paths and their field support for refinement.
The original Spider2 runs produced:
K = 4 is a reasonable default balancing path coverage and computational cost, rather than an optimal setting. The appendix reports sensitivity to K (Figure 4).
2. Basis of the uncertainty signal and refinement’s contribution
In §3.1, the score is computed from schema support across independently grounded paths using Eqs. (3)–(5), rather than an LLM’s self-reported confidence. Like an SQL engineer comparing implementations, it uses agreement and disagreement over schema requirements to prioritize investigation. It is a path-support heuristic, not a calibrated probability of correctness.
Multi-path grounding prioritizes coverage of alternative SQL implementations, trading precision for recall. With grounding already achieving high recall, UGAR focuses on improving selection quality. The Table 2 ablation shows higher precision and NSF on both datasets after adding UGAR.
In this additional control on all 517 examples, both agents share initial grounding outputs and the paper’s Qwen3.8, tools, and budgets. The generic agent uses the same tools for refinement without uncertainty guidance: it does not partition schema items into certain and uncertain sets based on cross-path support.
With equal turn limits, guidance reduces refinement tokens by 13.05%, evidence-tool calls from 567 to 424, and model calls from 952 to 750, with small observed gains in all four schema metrics. This shows improved schema quality at lower cost. We will add this experiment and analysis to the revised manuscript.
3. Recall, precision, and database size
Submitted Table 1 gives EviLink the highest Spider2 field SRR: 93.04% vs. APEX-SQL’s 80.85%, with lower NSP/NSF (26.04%/36.65% vs. 47.50%/58.64%). On BIRD, it has lower SRR than AutoLink (93.22% vs. 98.89%) but higher NSP/NSF (51.58%/63.60% vs. 11.39%/19.28%). Submitted Table 12:
EviLink leads APEX-SQL across sizes, most on large schemas (18.45 pp vs. 7.87 on medium), supporting its large-schema coverage advantage.