- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The manuscript proposes SENECA, a new entropy estimator for discrete distributions in the small-sample and under-sampled regime. The central idea is to estimate missing mass through a fixed-point equation that combines observed empirical label frequencies with an estimate of the number of unobserved labels, obtained from a support-size estimator. The resulting missing-mass estimate is then plugged into a Chao–Shen/Horvitz–Thompson-style entropy estimator. The authors provide existence and lasymptotical consistency results, extensive simulation experiments over 72 synthetic distributions, an application to 58 biodiversity collections, and an application to LLM uncertainty quantification.
Critical strengths
The methods presented in the manuscript demonstrated competitive performance in the simulation experiments.
Critical weaknesses
1 The manuscript provides a proof of the asymptotical consistency of its method. However it is well known that several commonly used entropy estimators possess asymptotical consistency. The manuscript did not discuss whether the method possesses more interesting statistical characteristics such as unbiasedness or sample complexity.
2 Uniqueness and convergence of the fixed-point estimator are not established. Lemma 1 proves existence of a fixed point of Eq. 5, but not uniqueness. Algorithm 1 uses Steffensen iteration from an initial value of 0.5. If multiple fixed points exist, it is unclear which one is selected and whether the iteration is guaranteed to converge to the intended solution.
In summary, there is room for further improvement in the theoretical solidity of the manuscript.
Questions for authors
The methods presented in the manuscript seem to perform less well on real data compared to that on simulated data. Could the author provide substantive analysis and explanation on this matter?
3 comments
Dear Reviewer SL65,
As the discussion period comes to a close, we are reaching out to gently follow up on this exchange. Your initial feedback was greatly appreciated, and we believe our efforts to address these concerns have led to an improved submission.
In short, we responded with regard to proofs of unbiasedness and sample complexity (W1) and added a proof of uniqueness of the fixed point solution (W2).
Please let us know if you feel that outstanding items persist, and we will do our best to address them in the time remaining.
Thank you,
Authors
Dear Reviewer SL65,
As we enter the final hours of the discussion period, we wanted to reach out once more to continue this productive dialogue, wherein the Reviewer pointed out several interesting topics. We are happy to answer any lingering questions, to the extent time allows.
We found the Reviewer’s feedback thoughtful and constructive, and we are grateful they lent their time to the consideration of our work.
All the best,
Authors
We thank Reviewer
SL65for their time spent reviewing our submission, as well as their constructive comments. We are encouraged that the Reviewer points out ”competitive performance” in ”extensive simulation experiments” and finds no substantive concerns about the paper’s soundness.We also appreciate the Reviewer’s patience, as we prepared a thorough response to accommodate as many of the Reviewers’ helpful comments as possible (our efforts proved a bit more time-consuming than initially anticipated). Our revised manuscript highlights substantive additions in blue. We have also added a small table of contents to the appendix section with auxiliary proofs, for the Reviewer’s organizational convenience.
We respond to the Reviewer’s comments below:
We agree that some entropy estimators (e.g., Plugin and Chao-Shen) are known to be asymptotically consistent (which we note in Section 6.2), but the results the Reviewer describes can be quite challenging to produce and are often not established in the works introducing the methods. For example, we are not aware of general proofs of consistency, unbiasedness, or sample complexity in the papers introducing the Chao-Shen, Chao-Wang-Jost, James-Stein, or Bonachela estimators. Consequently, empirical analyses are more typical, like the ones we perform. In Figure 8, we characterize the included entropy estimators' biases empirically, and we hope this is satisfactory to the Reviewer. We conclude by noting that, in general, unbiased entropy estimators do not exist.
This is a good point, and we agree that establishing the uniqueness of the fixed point is important. In the revision, we have identified a minimally restrictive upper bound on υ which allows us to prove uniqueness of the fixed-point solution on [0,1] (Thms. 1 and 2). After repeating our experiments with this upper bound, we discern no difference in the headline results arising from this update (e.g., Table 1 is identical).
We have added a paragraph discussing this topic to the Discussion. For convenience, we reproduce it below:
We reiterate our gratitude for the Reviewer's engagement, which has helped refine our work. Please let us know if any questions remain.