- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines when a joint-minus-conditioning kernel entropy can be interpreted as average within-condition diversity. It separates three sources of discrepancy: overlap in the conditioning representation, global spectral truncation, and finite-sample estimation. The main results include a sharp conditioning certificate, an exact equality characterization, recovery limits from aggregate scores, and counterexamples showing that global truncation can destroy the intended conditional interpretation.
Critical strengths
-
The paper proves that the joint-minus-conditioning (“conditional Vendi”) score does not generally equal average conditional entropy when condition features overlap, and gives a computable bound C(p,L) on the discrepancy.
-
The paper identifies global spectral truncation as an independent failure mode: even with orthogonal condition labels, a global top- budget can erase conditional entropy. The rank-budget law and finite Gaussian counterexample are particularly useful because they expose when truncated aggregation can be negative or misleading, and the blockwise tail-aggregation remedy preserves the conditional decomposition with explicit error accounting.
Critical weaknesses
-
The central certificate in Theorem 3.1 follows from data processing and entropy-of-mixing inequalities, while the blockwise tail construction uses the standard entropy grouping identity. These are useful applications, but their inclusion does not by itself establish a substantial theoretical advance. In my view, the more distinctive contributions are the quantitative overlap obstruction, the equality characterization, the aggregate-score recovery limit, and the truncation counterexample. I would encourage the authors to organize the theoretical narrative around these results and explain more precisely what they establish beyond existing kernel-entropy and quantum-information analyses.
-
The paper states that attainability of the conditioning bound and the minimax recovery lower bound rely on output ensembles that can realize mutually orthogonal states. However, a fixed CLIP, DINO, or other feature map generally restricts the admissible covariance operators. It remains unclear how closely realistic generators can approach the unrestricted worst case. A systematic comparison of the actual discrepancy with the certificate. For example, through the ratio ((G-H_{\mathrm{post}})/C(p,L))—would help assess its practical tightness. A stronger theoretical extension would characterize the gap under restrictions induced by the output representation or derive tighter certificates using additional output information.
-
In the Qwen confirmation study, the empirical entropy difference is approximately (0.110) nats, whereas the reported population difference interval is ([-2.423,2.642]). This is an informative negative result, but it leaves a substantial gap between finite-bank certification and reliable comparison of the underlying generators. The rank-based bias allowance alone is approximately (2.195) nats for (r=512) and (n=64), far exceeding the observed difference. The rare-tail impossibility result does not fully explain this limitation: it concerns a class without a common rank or tail restriction, whereas the linear-feature experiments have a known finite-dimensional representation. It would be valuable to establish whether the unresolved rankings primarily reflect insufficient data or conservatism in the analysis.
-
The 31 reversals among 112 GenEval specifications are correlated sensitivity analyses on the same output banks, rather than independent estimates of a failure rate. The initial tolerance certifies none of these reversals. The subsequent tighter tolerances demonstrate improved identification on the observed banks but do not establish prospective reliability. The Qwen confirmation study provides prospective evidence of reduced distortion, although the native temperature ranking is already correct. The fresh-image selection study is useful, but the held-out gain is approximately (0.0078) nats, direct grouped scoring achieves the same gain, and the population intervals remain inconclusive. A more decisive experiment would compare selection errors, abstention rates, and held-out gains across independent output banks under a fixed evaluation budget.
Questions for authors
The term “posterior diversity” should be used cautiously. Equality between the two entropy targets establishes agreement about a representation-dependent diversity functional. It does not establish posterior fidelity, calibration, or useful sample diversity. The manuscript acknowledges this distinction, but most experiments evaluate generator distributions without a known posterior reference. The title and application claims should remain consistent with that scope.
3 comments
response to authors
The paper establishes a solid theoretical account of conditioning distortion, recovery limits, and the global truncation problem for finite condition catalogues. But there are still some key concerns remain unresolved: population entropy rankings are still open, the empirical evidence is narrow and heavily controlled, and the finite-sample intervals are often too wide to support the very comparisons the paper is trying to justify. The counterexample to Jalali et al. may be important, but its assumptions and scope still need careful check.
Scope of the truncation counterexample
Thank you for distinguishing the theoretical result from the empirical evidence. We agree that our experiments do not establish population entropy rankings or tell us how often rankings change across generators.
On the counterexample, Section 4 and Example C.3 address a specific claim: Eq. (19) of Theorem 7 in Appendix B of Jalali et al. The prompt is equally likely to be 0 or 4, and the output is an independent draw from the same two points. Both kernels are Gaussian with bandwidth one, both feature maps have unit norm, and each prompt component has zero variance. With two eigenvalues retained and the omitted mass redistributed equally, the discrepancy is 0.693147 nats, while the stated upper bound is 0.026417 nats.
Independence makes the distinction particularly clear: before truncation, the joint-minus-prompt entropy equals the within-prompt entropy exactly. The discrepancy arises because retaining two eigenvalues of the joint operator is different from retaining two in each conditional operator. The example contradicts the truncated aggregation bound as stated; it does not contradict the untruncated score or establish how often truncation matters in practice.
Response to Reviewer MJLe
Your distinction between standard ingredients and the more specific results has guided the revision. The introduction now centres the contribution on the quantitative overlap obstruction, equality characterisation, recovery limits from an aggregate score, and the separate truncation obstruction. We identify data processing, entropy of mixing and tail aggregation as established tools used in this analysis.
1. Fixed representations and realised tightness. We agree that unrestricted attainability does not establish sharpness for every fixed encoder. We have now recomputed (G−H^group)/C across all 336 score rows of the original GenEval grid. These retain all three systems, both representations, both output kernels, seven prompt subsets and four conditioning kernels. Appendix K.1 (Table 17) reports absolute gaps and certificates alongside the ratios.
The certificate ranges from 0.064 to 3.799 nats in each stratum. DINOv2-linear ratios on the full 100-prompt bank range from 0.819 to 0.879, so the high ratios are not confined to small subsets. These descriptive results show that substantial fractions of the bound occur with the fixed empirical feature maps. They concern score magnitude, not improved ranking certification. Similar gaps can cancel in a score difference. The configurations share outputs and are not independent replications. A representation-specific sharpness theorem or tighter output-dependent certificate remains beyond the present results.
2. Population uncertainty and the role of the bias allowance. You are correct that the unrestricted rare-tail construction does not explain the finite-dimensional Qwen result. We have separated these points in the revision. At n=64 and r=512, the bias allowance is 2.195487 nats and the concentration radius is 0.168481 per temperature. The former contributes 86.7% of the difference-interval half-width. This identifies the dominant term in the interval width, without measuring the actual population bias or showing that more data are unnecessary.
The new retrospective Qwen curves use the first 1, 2, 4, 8, 16, 32 and 64 stored draws per prompt. The observed difference stays positive from 2 to 64, but both entropy estimates still increase from 32 to 64 draws, by 0.033791 and 0.050260 nats. No population interval resolves the ordering. The full-sample Vendi ratio is 1.116 with interval [0.089, 14.043]. Thus the new analysis locates the dominant allowance and shows sample-size sensitivity. It does not determine whether a sharper analysis would certify this comparison at the existing budget.
3. Decision reliability and analysis history. We retain the original 0-of-31 certification result and label the tighter GenEval tolerances as post hoc. Replaying the unmodified scores reproduces all 31 reversals among 112 correlated configurations, but this supplies no independent failure-rate estimate. The prospective Qwen study preserves the native ordering. The fresh-image illustration concerns one aggregate decision, a held-out gain of 0.007782 nats, and the same gain under direct scoring. It does not establish an advantage over that baseline or repeated-study reliability. We have not added the proposed independent-bank selection experiment, so selection errors, abstention rates and gains across independent studies remain unresolved.
4. Posterior interpretation. The abstract and setup now make explicit that posterior diversity is an application when the sampled conditional law is the specified Bayesian posterior. The generator experiments assess representation-dependent conditional diversity. Agreement between the entropy targets establishes neither posterior fidelity nor calibration or output quality. These scope statements also govern the revised practical recommendations.