- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper formulates conservative KBQA query repair as a risk-control problem and proposes a training-free acceptance gate based on the repairer's KEEP-sequence probability. A Clopper–Pearson bound is used to control the population regression rate under specified calibration assumptions. The formulation is useful, but the semantic meaning of the acceptance score and the methodological novelty beyond established risk-control techniques remain concerns.
Critical strengths
The paper explicitly accounts for the asymmetric costs of correcting errors and introducing regressions.
The acceptance rule provides a clearly stated finite-sample guarantee under its assumptions.
Critical weaknesses
- We appreciate the theoretical contribution (relative repair-gain objective and principled thresholding), but practical importance in the high-accuracy regime remains weakly supported: accuracy changes only from 93.96% to 93.80% (repair) and 94.44% (proposed method).
- Positive results are mainly for one repair model on KQA Pro; transfer settings show zero or negative net gain, supporting damage limitation rather than accuracy improvement.
- KEEP depends on model decoding probability, is validated mainly on Qwen3-14B-AWQ, and its advantage over the prompted judge is small; uncertainty and generalizability need clarification.
Questions for authors
Beyond applying an existing risk-control procedure, what is the novel algorithmic contribution?