- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ADOQ, which combines fixed class centers, normalized distances, and class-conditional margin ranks to learn a representation with prescribed chi-square radial targets for selective classification. Once the model and analytical regions are frozen, independent data certify regional occupancy and accuracy using DKW and binomial confidence bounds. Experiments on four image-classification datasets evaluate selective performance and regional reliability. While the training construction is interesting and the certification appears sound under the stated assumptions, the missing matched certification comparison, single-run evaluation, and differing training protocols leave ADOQ’s incremental benefit and its robustness insufficiently established.
Critical strengths
An interesting and clearly formulated training construction. The combination of fixed-center geometry, margin-based ordering, and prescribed radial targets provides a concrete approach to organizing classification difficulty on a common numerical coordinate.
Well-specified certification with evidence of reliable subsets. The regional guarantees appear technically valid under the stated independence and multiple-comparison conditions, without requiring an exact chi-square fit. The experiments demonstrate reliability stratification, and the paper transparently discusses distributional mismatch and operational limitations.
Critical weaknesses
The central incremental value is not established by the comparisons. The certification argument does not depend on ADOQ’s geometry or training objective. Strong frozen scores such as MSP or Mahalanobis distance can also define regions before certification and receive the same type of statistical guarantees. However, the paper lacks a comparison with these alternatives under matched certification sample budgets, confidence levels, region-family sizes, and construction/selection protocols. The SGR experiment uses a different confidence level and selection mechanism and is explicitly not a performance comparison. Thus, the reported reliable subsets do not establish the added value of the specialized representation learning.
The operational benefit of the prescribed scale remains unclear under substantial mismatch. Proposition 3 characterizes an ideal population target, rather than guaranteeing the distribution or reliability ordering of the learned operational score. On CIFAR-100, the reference KS statistic is 0.3900, while the outer region with nominal mass 0.1% contains 30.67% of test examples. This does not invalidate certification, but substantially weakens the nominal occupancy interpretation. The paper has not established what practical benefit this prescribed coordinate provides over ordinary fixed thresholds or quantiles constructed independently of certification.
The experimental design limits conclusions about robustness and attribution. Each main configuration uses a single training run, so the stability of the learned structure and benefits across retraining is unknown. ADOQ also uses 400 epochs, compared with 200 or 300 for several baselines, and Imagenette augmentation differs. These differences particularly complicate attribution on Imagenette, the only dataset where ADOQ outperforms MSP in both accuracy and AURC. On the other three datasets, its AURC is worse. Different training recipes and non-SOTA performance are not independently grounds for rejection, but without controlled comparisons and independent repetitions, the evidence does not establish a robust additional benefit that compensates for these performance trade-off.
Questions for authors
-
Under matched certification sample budgets, confidence levels, region-family sizes, and construction/selection protocols, what advantage does ADOQ provide over strong frozen scores? In particular, does it improve certified retention or reduce certification sample requirements?
-
What operational benefit remains when empirical occupancy differs substantially from the prescribed scale, and is that benefit supported by independent training repetitions and controls for training budget and augmentation?