- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Semantic entropy estimation typically involves (1) sampling times from a language model, (2) clustering the responses using a pairwise judge, and (3) estimating the entropy of the true distribution over clusters. In practice, this true distribution, its support size, etc. are not known, and several works have attempted to better estimate the functional of interest. Fewer works, on the other hand, are concerned with the pairwise judge (step 2). With this in mind, the authors study the minimax L2 risk of entropy estimation, where the judge/aggregator is noisy.
The paper itself is exceedingly dense, with somewhere in the vicinity of pages of proofs. There are several aspects the authors draw upon which are outside of my familiarity, and therefore it is intractable to thoroughly digest all these proofs in the time available, hence my confidence rating below. In my opinion, this kind of submission is better suited for a journal with longer review period. I am placing this comment here, because it does not necessarily pertain to the quality of the work presented and should not be treated as a weakness per se. That said, I do think it is quite crucial, as the paper is almost exclusively theoretical.
Critical strengths
S1: The topic is interesting and relevant. In fact, several things are undesirable about step 2 above (the pairwise judge) - for one, it is slow for large . For another, the judge may be bad or otherwise noisy (as the authors study), and if I remember correctly [1] observed that even humans did not fully agree on cluster assignments (and LLM judge agreed with human raters roughly as often as humans agreed with each other).
S2: Section 5.1, which identifies a region where consistent entropy estimation is impossible under a noisy judge, is interesting and the separation criterion seems sufficient to be a useful reference (i.e., the minimum separation for consistent estimation is not trivially small).
References
[1] Farquhar et al. "Detecting hallucinations in large language models using semantic entropy." Nature (2024).
Critical weaknesses
W1: Limited scope of applicability, for which semantic entropy calculated via (1) pairwise judge-based semantic aggregation, (2) where the true distribution over semantic clusters is drawn from a 3D probability simplex, and (3) where the judge's errors follow Eq. 1. I expect that this combination of constraints is quite rare, in practice.
- Regarding (2): property estimation in general is especially challenging when the sample is smaller than the support, and this basically unaddressed by this work (and explicitly excluded in, e.g., Theorem 2, Corollary 3, etc.)
- Regarding (3): Eq. 1 assigns the same clustering probability to all generations that truly do belong to the same cluster, which seems unrealistic. For instance, a judge based on an NLI model or good LLM should group together identical generations with probability at or near 1, but this would not be expected for more lexically distant, but still semantically equivalent, generations. Indeed, this is just a model, but it is incumbent on the authors to justify this model. Actually, it seems possible that this constraint could be more relevant outside the natural language generation setting (e.g., perhaps species identification has Bernoulli-esque noise).
W2: Abstruse writing throughout is a major obstacle, and this is independent of the comment I make in the Summary. For example, this statement from the abstract
"It reads the collision probability off the agreement density and the triple-collision probability off a centred statistic in the squared degrees, projects the pair onto the set of valid moments, and solves a cubic."
and this one from the introduction
"Its core is a path at constant agreement density, along which a Glauber identity and a blockwise Efron-Stein bound control the Fisher information of the graph despite the hidden meanings."
are extremely difficult to parse without inferring author intent or having already read the main body of the paper. In general, there are many unconventional, undefined, or otherwise confusing aspects throughout (e.g., Sections 4.2 and 5.1 allude to an unexplicated "Glauber identity" and Laplacian-K is only defined in Appendix G.3) that should be clarified in revision.
W3: I am cautious of some of the mathematical content, apropos of the caveat included in the Summary section. To start, it is not obvious to me that is an unbiased estimator of (L188). I suspect that may be a negatively biased, but consistent, estimator of , though I could be wrong. Can the authors elaborate on this? For simplicity, let's take a noiseless judge (i.e., , ) and . Also, let denote the set of integers in , which I am struggling to typeset here. Then , and, if I am correct, . For my curiosity, perhaps the authors can run a large number of Monte Carlo trials illustrating this scenario with some small values of , e.g., .
I have similar feelings elsewhere. I am not going to be exhaustive here, as the guidance suggests these explanations be brief, but I feel a point-by-point review of the proofs is warranted, and W2 makes this especially challenging.
Questions for authors
Please see the Weaknesses, where I have provided many details.
-
Can the authors please clarify which presented items are original and which are drawn from prior work? For instance, the related work says "The two identities we use appear in Ambroise & Matias (2012)" - which two identities (does this refer to Eq. 4)?
-
Can the authors please review their notation for conceptual overlap? E.g., (including with subscript) refers to both the probability simplex and the judge's separation).
-
Is Figure 2 correctly labeled?
2 comments
Response to Reviewer DXdR (2/2)
Details behind our comment (1/2).
The check of in your example. Equation (4) is evaluated literally from the degrees. "Exact" sums over every noiseless labelling. For the noisy channel it groups labellings by class size with their binomial weights and enumerates all graphs for each size. "MC" is the mean over independent trials, with its standard error in parentheses.
| noiseless, exact | noiseless, MC ( trials) | , , exact | , , MC ( trials) | |
|---|---|---|---|---|
| 4 | 1/4 | 0.25049 (0.00031) | 1/4 | 0.2465 (0.0032) |
| 5 | 1/4 | 0.25046 (0.00024) | 1/4 | 0.2526 (0.0021) |
| 6 | 1/4 | 0.24971 (0.00019) | 1/4 | 0.2517 (0.0016) |
| 7 | 1/4 | 0.24989 (0.00016) | not enumerated | 0.2504 (0.0013) |
| 8 | 1/4 | 0.24988 (0.00014) | not enumerated | 0.2501 (0.0011) |
| 9 | 1/4 | 0.25017 (0.00013) | not enumerated | 0.2498 (0.0009) |
| 10 | 1/4 | 0.25003 (0.00011) | not enumerated | 0.2508 (0.0008) |
The two quoted sentences in plain terms.
The abstract sentence. The estimator uses two summary statistics of the verdicts. Subtracting from the fraction of pairs judged equivalent and dividing by estimates , the probability that two answers share a meaning. A corrected average of the squared degrees, divided by , estimates , the probability that three answers share a meaning. The degree of an answer is the number of answers it is judged equivalent to. Noise can put outside the moment pairs of distributions on at most three meanings, and we move it to the nearest such pair. With at most three meanings the proportions are then the roots of a cubic, and their entropy is the estimate.
The introduction sentence. We compare two distributions with different entropies and the same expected number of agreeing pairs, joined by a path of distributions that keeps that expectation fixed. The Fisher information of the graph along the path bounds the total variation distance between the two graph laws, which limits how reliably any estimator can tell them apart. What we called the Glauber identity (Lemma D.8) is Bayes' rule for one answer: given the graph and all other meanings, multiply each meaning's prior weight by the likelihood of that answer's verdicts if it had meaning , and normalize. With it, Lemma D.10 expresses the graph score (the derivative of its log-likelihood along the path) as the conditional mean, given the graph, of an explicit mean-zero function of the graph and meanings. This function's second moment bounds the Fisher information. Efron–Stein bounds this second moment by half the sum of the expected squared changes when one edge's randomness or one answer's meaning is resampled at a time.
A reading route for the proofs.
- The estimator's guarantee (Theorem 2). Appendix B, pages 12 to 19, self-contained.
- Whole-grouping separation. Under Assumption 1, , , Theorem 2 gives uniform entropy consistency. Proposition 8 rules out recovery up to relabelling for fixed non-point-mass (Appendix E.2, about one page).
- The per-answer statement with an oracle (Proposition 7). This also uses the posterior identity and several auxiliary bounds from Appendix D.
- No uniformly consistent estimator unless (Proposition 6). Appendix C, pages 19 to 27.
- The matching lower bound (Theorem 4). All of Appendix D, pages 27 to 45. Figure 4 is its roadmap.
The appendix also reports numerical checks of selected pieces. Over 216 simulated combinations, the means of and of are within 2.3 standard errors of their predicted values (Appendix G.4). These checks test the implementation and selected identities, not the uniform bounds, and they do not replace reading the lower-bound proof. We would be glad to go through any step you would like checked.
Measured bias under the real judges. Two equivalence rules (bidirectional entailment, or the default of the released code of Farquhar et al., 2024) and two template sets (all 12, or the 7 without first-person or modal markers) give 16 configurations for the four entailment models. With the channel calibrated on 40 other questions, the exact biases of under fixed verdicts and uniform template sampling, averaged over the six assignments of each of the four distributions of Table 1 to the candidates, range from to after averaging over questions and from to for individual questions. A language-model judge, Qwen2.5-14B-Instruct with the entailment prompt of that released code, behaves differently. Across its four configurations the test-set mean agreement rates are 0.932 to 0.966 for the correct candidate and 0.075 to 0.127 for the same distractor. This is class-dependent behaviour, which the Limitations name as not covered.
Response to Reviewer DXdR (1/2)
W3. With the true channel, P3 is exactly unbiased for every n≥3. Write di−q(n−1)=∑j=i(Aij−q) and expand the square. As Aij∈{0,1}, the diagonal terms sum to di(1−q)2+(n−1−di)q2, exactly what Equation (4) subtracts. So P3 is the average of (Aij−q)(Aik−q)/Δ2 over ordered triples of distinct answers. Given the meanings, the two verdicts are independent with E[Aij−q∣Z]=Δ1{Zi=Zj}, so each term has mean P(Zi=Zj=Zk)=P3 (Lemma B.1).
Thank you for the checkable example (q=0, Δ=1, θ=(0.5,0.5,0)). There di=nZi−1, with nc the number of answers of meaning c, so P3=∑cnc(nc−1)(nc−2)/(n(n−1)(n−2)). Since n1,n2∼Bin(n,21) have E[nc(nc−1)(nc−2)]=n(n−1)(n−2)/8, EP3=41=P3. Your (i−1)(i−2)/((n−1)(n−2)) is the contribution of one answer whose meaning is shared by i answers. P3 averages it over all n answers: at n=4 it takes the values 1,41,0 with probabilities 162,168,166, with mean 41.
Evaluating Equation (4) literally in your example, 106 trials per n=4,…,10 give means 0.24971 to 0.25049, all within two standard errors of 41. Exact enumeration gives 41 at each n, also at n=4,5,6 with q=0.3, Δ=0.2. The table is in our comment (2/2) and will go into the appendix.
Proofs. The qualitative separation uses only Theorem 2 (Appendix B) and Proposition 8 (Appendix E.2, about one page). Only the matching lower bound and the per-answer statement of Proposition 7 need Appendix D. Our comment (2/2) gives a reading route.
Q1. Yes: Equation (4) uses E[Aij]=q+ΔP2 and E[(Aij−q)(Aik−q)]=Δ2P3. Ambroise and Matias (2012) give these identities and estimate unordered proportions without recovering labels. Liu (2026) bounds the entropy error via clustering, with no lower bound, and projection before inversion is prior (Wu and Yang, 2020). We prove matching entropy-risk bounds as the judge weakens, the exact consistency threshold, and entropy estimation where recovery is impossible (Theorems 2, 4 and 5, Propositions 6 to 8). The entropy bound includes the simplex boundary. We will state this split in the related work.
Q2, Q3. We will write the simplex as S3, reserving Δ for the judge, and give the class counts of Appendix D their own symbols. Figure 2 is correctly labelled but under-explained: its axes are e2=(1−P2)/2 and e3=(1−3P2+2P3)/6, and the triples printed in the plane are the distributions θ at those points, not coordinates. We will say so in the caption and label the points θ=(1,0,0) etc.
W2. We agree and will replace both quoted sentences. The estimator infers the probabilities that two or three answers share a meaning, then computes the entropy from them. The lower bound compares distributions with different entropies but equal expected edge counts, bounding the Fisher information along a path between them. We will state the Glauber identity (Lemma D.8), the conditional law of one meaning given the graph and all other meanings, on first use in Section 4.2. Figure 1's caption will define Laplacian-K (Liu, 2026), which groups answers by the eigenvectors of the K smallest eigenvalues of D−A, D being the diagonal degree matrix and K the true number of meanings.
W1. Scope. Equation (1), as in Liu (2026), isolates the cost of not observing the meanings. With observed labels the entropy on a fixed alphabet has minimax rate 1/n, and seeing the labels only through a weak judge costs a factor n2γ−1/2 for Δ=n−γ, 1/2<γ<3/4 (Section 2). We have no matched rate when the support can exceed n, and we have not established all three constraints jointly for real judges. Under the stated hypotheses, the lower bounds extend to any support bound K≥3 and to larger classes containing Equation (1) with the same graph data and entropy target. Inclusion alone does not extend the upper-bound guarantee.
We tested four entailment models on 80 SciQ questions, each with three candidate words in 12 templates. Meaning is candidate identity, ignoring confidence wording. All four judge identical strings equivalent. With all 12 templates, mean bidirectional agreement on distinct wordings of the same candidate ranges from 0.639 to 0.943. So the entailment models show the contrast you describe, for wording around a fixed word rather than distant paraphrases. Across 16 configurations (4 models, 2 rules, 2 template sets), calibration on 40 other questions gives the pooled diagnostic nΔ^2≥4.58 at n=10, far outside the matching lower-bound regime (nΔ2≤c1<0.01). These measurements do not validate the theorems and will go into the appendix.