- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free confidence estimation method for KGQA by sampling PPR-guided paths, assigning terminal LLM-judge weights, and aggregating normalized weights by answer.
Critical strengths
The paper explicitly distinguishes path quality from answer correctness.
The answer-level aggregation mechanism is simple and does not require task-specific training.
Critical weaknesses
- Posterior mass is not necessarily a measure of answer correctness.
The method aggregates judge-weighted paths into an answer-level confidence score, but the judge does not observe the extracted candidate answer.
- Answer selection is inconsistent with answer aggregation.
The method returns the answer associated with the highest-weight individual particle rather than the answer with the largest aggregated mass.
Questions for authors
What assumptions justify interpreting judge-weighted path mass as answer correctness when the judge never evaluates the extracted candidate answer?