- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the KakeyaShare procedure for simultaneously estimating bounded loss averages over directional tubes in a three-dimensional input chart by grouping locations according to exact tube memberships. The main contribution lies in combining incidence-based stratification, resource allocation, and volume calibration, which yields simultaneous risk coverage guarantee.
Critical strengths
The sharing construction is intuitive, and the integer allocation is optimal for its stated sampling objective. The risk coverage argument explicitly accounts for geometry-estimation error, which is essential for a valid certification method.
Critical weaknesses
1. Optimization criterion and novelty. The proposed allocation scheme is optimal for the objective in Equation (7), but not necessarily for the final certified interval width, which also involves geometric uncertainty. Moreover, the resource allocation is closely related to classical stratified sampling and Neyman allocation, which limits the methodological novelty.
2. Practical efficiency. The query savings are reported only against Monte Carlo with Hoeffding intervals. It is unclear whether the saving persists when Monte Carlo uses tighter confidence bounds. It would be better to establish whether fewer model calls actually reduce total runtime after accounting for the additional geometry computation.
Questions for authors
1. (Optimality) Why minimize the sum of squared sampling radii in Equations (6)-(7) rather than the final or , which include geometry uncertainty? How would these alternative criteria affect the core selection and budget allocation?
2. (Theoretical guarantee) Theorem 1 establishes the simultaneous risk coverage, and Equation (13) bounds the sharing gain under the full partition. But there seem to be no theoretical conditions on , , or that ensure a narrower certified radius than IMC at the same query budget.
3. (Simulation) In Table 2, how would incorporating geometry uncertainty into core selection change the selected core and the final ?
4. (Presentation) A small concern about Figure 2. It is overly dense, which makes the key idea hard to understand.
3 comments
Response to Reviewer MgcE (Q3, Q4)
Q3. How would including geometry uncertainty affect core choice?
The existing ablation already reveals the direction of this effect. At , the fixed 360-atom core yields final , while the sampling-selected 450-atom core yields : the smaller core is approximately 0.9% better on the full certified objective among these two evaluated candidates (Table 2). Increasing the core can reduce the sampling term while increasing the number of calibration categories and thus the geometry term. Revised Appendix A.2 now sets out how a geometry-aware design-stage objective could include both terms while keeping final calibration independent. This is evidence of the trade-off, not a claim that a new global core optimizer has already been implemented.
Novelty, tighter confidence bounds, runtime, and Q4
Novelty lies in the statistical unit and the valid end-to-end construction, not in claiming to invent Neyman allocation. Mixture methods reuse queries but retain random directional membership and importance weights inside each directional observation. KakeyaShare instead estimates one bounded mean per exact-membership atom, then propagates that mean into several targets using independently calibrated geometric coefficients. Budget-aware compression and exact joint allocation make this representation executable; Theorem 1 and Propositions 1–2 establish its coverage and finite-budget behavior.
The comparison includes Union-HT, DM-MIS, OptDM-MIS, and Multiplicity-Neyman, not just IMC (Table 1; Appendix C.2). We also take the tighter-bound question seriously: the revised Appendix A.2 explicitly reports that empirical-Bernstein IMC can yield narrower intervals than our Hoeffding-based certificate (Appendix D.5). This does not invalidate our certified comparison, but it precisely identifies the boundary of the claim: we establish an advantage for the stated simultaneous, range-based Hoeffding certificate, not dominance over every variance-adaptive method. The reported empirical-Bernstein offline radii have not been separately accompanied by a coverage experiment.
For efficiency, we do not equate fewer black-box calls with unconditional wall-clock speedup. Table 3 reports equal-query execution overhead; Appendix D.6 provides an explicit cost-amortization condition that includes geometry setup and rejection. Where model calls are expensive and canonical geometry is reusable, avoided evaluations can offset that work.
Finally, Figure 2 has been completely redrawn (p. 4) to present the core idea in three direct steps—find shared atoms, estimate a bounded atom mean, reuse it across directions—and the opening now explains an explicit risk-screening decision.
We believe the two newly proved propositions directly resolve the principal theoretical questions, while the revised figure and comparison discussion substantially strengthen the novelty and presentation. We respectfully ask the reviewer to reconsider the recommendation in light of these changes.
I appreciate the authors' clarifications and the additional allocation guarantee. However, my concern about the methodological contribution remains. The new guarantee assumes a fixed core, while the core selection still optimizes sampling error without accounting for geometry uncertainty in the final width, and the allocation scheme is closely related to the classical Neyman allocation. I remain unconvinced of the methodological advance, so I would prefer to retain my original rating.
Response to Reviewer MgcE (Q1, Q2)
We sincerely thank the reviewer for identifying the core technical questions and for recognizing the validity of the geometry-aware coverage argument and the exact integer allocation. Your review prompted us to strengthen the theory rather than simply restate the original claims. The revised paper adds two formal results that directly address Q1 and Q2 (Propositions 1–2, p. 6; complete proofs in Appendices B.8–B.9), while preserving the original algorithm and experimental protocol.
Q1. Why minimize the squared sampling radii rather than the final certified radius?
Equation (7) is an intentional optimization target: it measures how one accepted observation contributes across all directional estimates and has an exact integer minimizer, not a rounded continuous heuristic. The revised Proposition 2 now quantifies what this choice costs relative to optimizing the final, geometry-calibrated RMS radius.
Fix the core, calibrated reconstruction matrix, and geometry-error vector. Let nΦ minimize Eq. (7) and let n⋆ minimize the final R2 over the same feasible integer allocations. Write S⋆ for the RMS sampling radius attained by nΦ and G for the RMS geometry-error term. Then
\leq S_\star+G-\sqrt{S_\star^2+G^2} \leq\min\{S_\star,G\}.$$ Moreover, $$\frac{R_2(n_\Phi)}{R_2(n^\star)}\leq\sqrt{2}.$$ This is a uniform approximation guarantee for the final certified RMS radius, not merely a statement about a surrogate. In particular, the additive gap vanishes as the geometry error vanishes. We distinguish this fixed-core $R_2$ guarantee from optimizing the maximum-direction radius $R_\infty$, which is a separate objective. The proof is in Appendix B.9. Q2. When does KakeyaShare provably outperform IMC at equal budget and confidence? We now provide the requested finite-budget sufficient condition for the final certified radius, rather than relying on the ideal full-partition gain of Eq. (13). The revised Proposition 1 fixes the design-selected core, assumes $N > K+D$, $D\mid N$, and feasible exact stratum sampling, and incorporates its actual incidence benefit $$B_C=\sum_{S\in C}\bigl(\lVert w_{S:}\rVert_1-\lVert w_{S:}\rVert_2\bigr).$$ For total failure allowance $\delta=\delta_s+\delta_g$, define $$\alpha=\sqrt{\frac{\log(2D/\delta_s)}{2D(N-K-D)}},\qquad \beta=\frac{\sqrt{K}+\sqrt{2\log(D/\delta_g)}}{\sqrt{G_g}},$$ $$r_0=\sqrt{\frac{D\log(2D/\delta)}{2N}}.$$ If $$\boxed{\alpha\bigl(D-B_C+D\beta/2\bigr)+\beta/2 < r_0,}$$ then the final, unclipped KakeyaShare RMS certified radius is strictly smaller than IMC's Hoeffding radius at the same total budget and confidence, with conditional probability at least $1-\delta_g$ over the independent geometry-calibration stream. Theorem 1 continues to guarantee simultaneous coverage at level $1-\delta$. Appendix B.8 proves the bound, explicitly accounts for integer initialization and weight-calibration perturbations, and gives a sufficient lower threshold on $G_g$. Crucially, this is a sufficient, not necessary, condition; it does not assert favorable overlap for every tube family. The observed calibrated main-geometry result is already compelling: at $D=12$, $N/D=512$, and common 95% simultaneous confidence, KakeyaShare gives $R_2=0.063860$ versus IMC's $0.077647$, a 17.8% narrower RMS certificate. The interpolated matched-radius crossing is 340.5 versus 512 accepted queries per direction, a 33.5% reduction (Appendix C.7). We deliberately distinguish this observed comparison from numerical verification of the conservative true-volume sufficient condition, for which the reported aggregate tables do not provide all inputs.