- Rating
- 2(−1)
- Confidence
- 3
- Informativeness
- 2(−1)
- Presentation
- 1(−1)
- Soundness
- 2
Read review
Paper summary
The paper studies sign identification when preference categories are selectively reported and nonresponse is observed. It relates nuisance-profiled KL information to the first likelihood derivative that survives nuisance cancellation. For an exponential-tilt score model with degree- response-logit nuisance, it derives an exact support threshold, an explicit nominal information coefficient, and finite-gap alias radii. It also studies protocol design under joint validity uncertainty, reference-dependent decisions, and a quadratic–quartic crossover for a symmetric two-protocol product class. The evidence consists of analytic proofs and reported numerical certificates; the sample-complexity results are lower bounds.
Critical strengths
S1. An informative exact calculation of nuisance-induced information loss. The interpolation threshold and surviving orthogonal-polynomial coefficient explain why adding a scored category can change the nominal experiment from exact aliasing to higher-order information. The conditioning results also distinguish meaningful additional resolution from a nearly vanishing category or nearly coincident score. These are concrete insights beyond a general claim that richer feedback helps.
S2. Careful treatment of joint validity uncertainty. The distinctions between arbitrary joint sets, convex sets, rectangular budgets, and the symmetric product crossover are consequential. In particular, the examples showing that the same marginal error budgets can yield different information, and that a correlated mechanism can require three protocols, prevent an unjustified generalization of the pair-design formula. The distinction between ambiguity and an opposite unique decision is also valuable.
Critical weaknesses
W1. Nonidentifiable correctness classes limit the fixed-confidence interpretation. Definition 21 requires correctness at every unique-decision model in the class. This can already fail in the one-protocol class of equations (80)–(81), without response-error thickening. Take the allowed baseline and , and set for small . Both and belong to the class because their nuisance coefficients approach zero. The identity gives , including nonresponse, although the two models require opposite decisions. Thus no uniformly -correct procedure exists on this class for , even above the scored-support threshold. This does not contradict the information calculation at the nominal model .
The same class-wide issue affects the two-protocol uncertainty classes. Fix any row of equation (267) with . A model with and zero nuisance and deviations is exactly matched under both protocols by a model with , , and
\bigl(2ty_j+\psi(-t)-\psi(t)\bigr).$$ Equations (272)–(273) give equality of the complete laws, including nonresponse. These deviations approach zero with $t$, so both models belong to the same class for sufficiently small $t$, despite requiring opposite decisions. No uniformly $\delta$-correct procedure therefore exists on that row for $\delta<1/2$, even when the selected nominal instance has positive information. Shrinking $\gamma$ across rows does not remove this problem within each positive-radius row. Corollary 80's conditional lower bound consequently has no class-correct procedure to which it applies in these cases. The paper explicitly claims lower bounds, and these observations leave its nominal KL calculations and conditional inequalities intact. The concern is the fixed-confidence interpretation of equations (10), (20), and (38), including Corollary 80, on the stated correctness classes. Positive information at the selected nominal instance does not make class-wide identification feasible on those classes. **Update.** I agree with reviewer JrQD and 7JEX about the presentation issues. I lower my rating from 3 to 2.Questions for authors
Clarification concerning W1: Is “higher-order identification” intended only at the nominal response baseline? Which correctness class is intended for the fixed-confidence conclusions, and how does it exclude the opposite-decision aliases in W1, including the example without response-error thickening?