- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The authors study LLM uncertainty estimation and whether/when self-agreement of repeated model responses can support valid accuracy claims, in the setting where training and deployment inputs/prompts come from different populations.
They present and prove a number of theoretical results. The paper contains no experimental evaluation.
Critical strengths
- The studied problem (whether self-agreement can support a valid deployment accuracy claim under population/prompt shift) is interesting and important.
Critical weaknesses
-
At least to me personally, I found the paper extremely difficult to follow and understand (I found Section 2 and the start of Section 3 OK, but the rest of the paper very difficult).
-
This is an entirely theoretical work (without any empirical case study or experimental results whatsoever) and its actual practical relevance/usefulness is highly unclear (does this give any concrete guidance or tool beyond just generic common sense advice of "self-consistency should not be assumed to guarantee accuracy" or similar in practice?).
Questions for authors
Questions/suggestions:
-
Could you clarify what the main concrete contributions of the paper are (I struggled to understand point 1 to 4 in the Introduction)? As someone interested in estimation and evaluation of LLM uncertainty, what are the main things I should take away from this paper? Why is your work and these theoretical results important for me? How could/should I use this?
-
Is there really no type of experimental evaluation you could add to the paper? Or at least some kind of simple illustrative example?
Minor things:
- "Definition 1 (Accuracy agreement and agreement bias)" --> "Definition 1 (Accuracy, agreement and agreement bias)"?
- You use Definition 1, Lemma 2, Theorem 3, Theorem 4, Corollary 5, etc. I.e. one global numbering for all categories. Could it not be better to instead do Definition 1, Lemma 1, Theorem 1, Theorem 2, Corollary 1, etc?
Rating justification:
This seems like solid work to me and the authors seem very knowledgeable, but I just found the paper very difficult to follow/understand, and its practical relevance/usefulness is highly unclear to me. "I am familiar with this topic and its relevant literature" is correct for me, but I'm not overly confident in the review itself.
Summary of Revisions in Response to the Reviews
Dear Area Chair and Reviewers,
Thank you for your careful reading and constructive feedback. We appreciate the recognition of the importance of assessing deployment accuracy under prompt shift, the sharp characterization of worst-case ambiguity under partial source coverage, and the analysis of dependence among response pairs that share a generation.
In response to your feedback, we have made the following revisions.
Please refer to the latest revised PDF for the full set of changes, highlighted in blue, and for the experimental protocols and results.