- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a quantum–classical model for selecting among candidate SQL queries or visualization specifications generated by an LLM. It represents candidates as relational graphs and uses structural differences to drive joint quantum interactions. Training rewards the total probability assigned to valid candidates, although inference returns only one answer. Evaluation on Spider and nvBench includes baseline comparisons, ablations, measurement and noise analyses, and a small hardware study. The contribution is primarily a candidate reranking method rather than a mechanism for clarifying user intent.
Critical strengths
-
The model explicitly compares structural differences between candidates. This is a useful design choice for SQL and visualization outputs, where plausible candidates may differ in a join, grouping, or filter.
-
The evaluation goes beyond ideal simulation. Finite-shot, noise, and small-scale hardware experiments examine whether the learned candidate rankings survive execution constraints. These results help assess the gap between the proposed model and its behavior on a physical device.
Critical weaknesses
-
The formulation does not adequately address ambiguity in user intent. When candidates represent different plausible interpretations, selecting the highest-probability output does not establish what the user intended. The system neither asks for clarification nor explains its chosen interpretation. Benchmark agreement alone therefore provides limited evidence of practical ambiguity resolution. The paper should distinguish equivalent expressions of one intent from genuinely different intents and establish when automatic selection is appropriate.
-
The practical cost of the proposed approach remains unclear. The paper reports theoretical complexity and logical quantum resources, but not training time, inference latency, or measured resource use relative to classical baselines. Without these measurements or comparisons under equal compute budgets, it is difficult to assess whether the quantum–classical architecture offers a worthwhile practical trade-off.
Questions for authors
-
When candidates represent different plausible user intents, how does your method avoid returning an answer based on the wrong interpretation, given that it neither asks the user for clarification nor explains the interpretation it selects?
-
Can you report training time, per-example inference latency, and measured resource use for JointQ-Multi and the classical joint baseline under comparable conditions? Does the performance advantage persist under matched compute budgets?
3 comments
Response to Critical Weaknesses (CONT.)
W2. Practical computational cost and performance trade-off.
We thank the reviewer for emphasizing the importance of practical computational efficiency alongside predictive performance. We agree that theoretical complexity and logical quantum-resource estimates alone are insufficient to establish a favorable cost-performance trade-off. We have expanded the resource analysis in Section 5.3 and clarified the computational limitations in Section 5.4.
Our evaluation compares JointQ-Multi with Classical Joint-Set using identical cached candidate pools, graph-input protocols, disagreement features, and validation-based training procedures. JointQ-Multi employs three candidate-register qubits, four feature-register qubits, two reasoning layers, and five disagreement categories. With , this corresponds to seven logical qubits, ten candidate pairs, and 100 logical interaction factors.
Table R1. Performance and computational-resource comparison.
| Metric | Classical Joint-Set | JointQ-Multi |
|---|---|---|
| nvBench EM (%) | 60.1 ± 0.4 | 62.5 ± 0.2 |
| Spider EM (%) | 71.2 ± 0.3 | 73.4 ± 0.2 |
| Training time, nvBench (min) | 48 | 125 |
| Training time, Spider (min) | 65 | 168 |
| Inference latency (ms/example) | 9.2 | 28.5 |
| Peak GPU memory (GB) | 2.1 | 2.8 |
| Maximum candidates | 5 | 5 |
| Trainable interaction coefficients | 10 | 10 |
| Candidate-register qubits | N/A | 3 |
| Feature-register qubits | N/A | 4 |
| Total logical qubits | N/A | 7 |
| Candidate pairs () | 10 | 10 |
| Logical quantum interaction factors | N/A | 100 |
| Hardware evaluation shots/example | N/A | 1,024 |
The reported measurements demonstrate an accuracy–cost trade-off. JointQ-Multi improves EM by 2.4/2.2 percentage points on nvBench/Spider but requires approximately 2.6 times longer training. Its inference latency is 28.5 ms/example, compared with 9.2 ms/example for Classical Joint-Set, representing approximately 3.1 times higher latency. Peak GPU memory also increases from 2.1 GB to 2.8 GB. These results indicate that the predictive improvements incur additional computational overhead.
Importantly, the reported training and inference costs reflect the evaluated simulation-based configuration, not end-to-end physical quantum hardware execution. Matching the number of interaction coefficients also does not establish equivalent computational budgets. We therefore make no claim of quantum speedup or superior efficiency under matched compute.
We have clarified these distinctions in Sections 5.3–5.4 and Appendix D. Our contribution concerns the empirical effectiveness of disagreement-conditioned quantum candidate reasoning, while establishing practical hardware-level advantages requires further evaluation under matched resource budgets.
Response to Questions
Q1. How does the framework avoid returning a candidate based on the wrong user interpretation without clarification or explanation?
We appreciate this question and agree that distinguishing candidate ranking from user-intent recovery is important. Our framework selects the highest-ranked structured candidate using relational representations and disagreement-conditioned joint interactions. However, the selected candidate is not guaranteed to correspond to the user's intended interpretation when multiple semantically distinct intents are plausible.
As clarified in Sections 3.1 and 3.4, structurally different candidates do not necessarily represent different user intents. Some correspond to alternative SQL or visualization formulations with benchmark-verified equivalent semantics, whereas others may reflect genuinely competing interpretations. The multi-valid objective rewards probability assigned to verified alternatives rather than penalizing all outputs other than a single reference. However, it does not resolve uncertainty about unobserved user preferences. We therefore clarify that our method performs ambiguity-aware candidate selection rather than interactive intent disambiguation. The current implementation neither asks clarification questions nor generates explanations of the selected interpretation.
For applications requiring greater assurance, candidate-probability margins or calibrated uncertainty estimates could help identify cases requiring user clarification. Such mechanisms would require separate evaluation and are not part of the present framework. Accordingly, our results concern candidate selection under benchmark-specific validity criteria, not guaranteed recovery of user intent.
Q2. Can you report training time, inference latency, and measured resource use under comparable conditions? Does the advantage persist under matched compute budgets?
We agree that practical resource comparisons are essential for evaluating deployment trade-offs. We have expanded Section 5.3 and summarized the measurements in Table R1 of our response to W2. JointQ-Multi requires 125/168 minutes of training on nvBench/Spider, compared with 48/65 minutes for Classical Joint-Set. Its inference latency is 28.5 versus 9.2 ms/example, and peak GPU memory is 2.8 versus 2.1 GB.
The default quantum configuration uses seven logical qubits, ten candidate pairs, and 100 logical interaction factors. The physical-device study evaluates 120 examples with 1,024 shots each, totaling 122,880 measurement shots. These hardware measurements are separate from the reported simulation-based inference latency and do not represent end-to-end physical-device execution time.
JointQ-Multi achieves 2.4/2.2 EM-point improvements over Classical Joint-Set on nvBench/Spider under the evaluated conditions. Both methods use identical cached candidate pools, relational input protocols, and validation-based hyperparameter selection. However, these experiments do not establish that the improvements persist under identical training-time or inference-time computational budgets.
We therefore distinguish the controlled accuracy comparison from an equal-compute efficiency claim. A fully matched resource evaluation would additionally account for candidate generation, graph preprocessing, quantum simulation, circuit compilation, hardware execution, and tuning effort. We have clarified these limitations in Sections 5.3–5.4 and Appendix D. Our contribution concerns the empirical effectiveness of disagreement-conditioned quantum candidate reasoning rather than computational speedup.
Response to Critical Weaknesses
Thank you for recognizing the usefulness of structural disagreement modeling and the finite-shot, noise, and hardware evaluations. We appreciate the concerns regarding user-intent ambiguity and computational costs and have revised the manuscript accordingly.
W1. Distinguishing benchmark equivalence from genuine user-intent ambiguity.
We agree that selecting the highest-probability candidate does not necessarily establish the user's intended interpretation, particularly when multiple semantically distinct interpretations remain plausible. We have clarified this distinction in Sections 3.1, 3.4, and 5.4.
Our framework addresses benchmark-verified candidate selection, rather than interactive user-intent recovery. We distinguish between (i) structurally different expressions with equivalent benchmark-verified semantics and (ii) genuinely different interpretations of an underspecified request. The multi-valid objective avoids penalizing alternative verified outputs but does not independently determine the user's intended meaning. For Spider, verified alternatives must satisfy both canonicalized structural compatibility and execution agreement with the reference SQL query, as detailed in Appendix F.2. These alternatives should therefore not automatically be interpreted as independently verified user intents.
We have clarified that the proposed architecture is a joint candidate-ranking mechanism, not an interactive clarification system. Its selected output represents the highest-ranked candidate under benchmark supervision rather than guaranteed recovery of user intent.
Automatic selection may be appropriate when a single structured output is required and the remaining uncertainty is acceptable. When distinct interpretations could lead to materially different or consequential outcomes, user clarification would be preferable. A probability-margin-based clarification mechanism is a possible extension but is not part of the evaluated system. These revisions clarify the scope and limitations of our contribution without changing the underlying formulation.