acceptodds
Under review as a conference paper at ICLR 2027

When Are Classes Cones? Scale Invariance, Linear Sufficiency, and Conic Decision Geometry

Abstract

Representation-learning methods implicitly choose a geometry for each class. We derive conic decision geometry from positive-scaling invariance and affine Bayes realizability: invariance alone gives cones, while an invariant affine readout admits a bias-free representation and polyhedral convex decision regions. Support containment additionally requires separability, and positive angular margin rules out class hulls containing a line. Under a shared-concentration von Mises–Fisher model with equal priors, the readout is Bayes-optimal and pairwise Bhattacharyya overlap yields a closed-form error-bound objective. On CIFAR-100 with 15% labels, a five-seed factorial study separates the architecture and objective effects: the full method reaches 71.2% accuracy and exceeds homogeneous cosine cross-entropy by 4.2 percentage points (paired 95% CI [4.05, 4.35]). Across 30 conditions, a finite-sample geometric quantity predicts error with cross-validated . The experiments also show improved shifted calibration, density-based OOD detection, and gain agreement of 100% over the tested range. These findings support the mechanism without establishing exact vMF fit, universal OOD optimality, or a positive interaction between architecture and objective.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
scale invariance, conic decision geometry, von Mises-Fisher, hyperspherical representations, positive homogeneity, bias-free networks, Bhattacharyya bound, calibration, out-of-distribution detection, label efficiency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zdiyOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks when conic geometry is justified for classification. It separates positive-scaling invariance, affine Bayes realizability, and zero Bayes error, showing what each assumption contributes to the geometry of decision regions and class supports. Under an additional shared-concentration vMF model with equal class priors, the authors derive a Bhattacharyya error-bound objective and a density-based rejection score. The proposed implementation uses a positively homogeneous network and class mean directions as the readout. A factorial experiment on CIFAR-100 with 15% labels reports 71.2% accuracy, compared with 67.0% for homogeneous cosine cross-entropy. Further experiments examine error prediction, gain invariance, calibration, and class-holdout OOD detection.

Critical strengths
  1. The distinction between invariant decision regions and class supports is the strongest part of the paper. Scale invariance alone does not give convexity, and support containment needs an additional separability assumption. The counterexample and the bias-removal argument make these distinctions concrete. I also found the connection between positive angular margin and pointed class hulls useful. The individual mathematical tools are fairly classical, but putting the assumptions together gives a clearer account of when a conic model is appropriate.
  2. The architecture-by-objective comparison is well chosen. The 4.2-point gain over homogeneous cosine CE is more informative than the larger gain over the standard network, because it shows that the proposed training objective contributes even after the architectural change. The standard-architecture comparison points in the same direction. These results provide a practical reason to consider the objective, beyond the geometric characterization.
Critical weaknesses
  1. The experiments do not yet isolate what is gained specifically from minimizing the Bhattacharyya bound. The factorial study separates architecture from objective, but the comparison with cosine CE may also change how class directions and concentration are estimated. The proposed readout uses class resultants without separately learned classifier weights. A matched vMF likelihood objective using the same resultants and concentration estimator would be particularly informative here. Section 3 correctly distinguishes bound minimization from likelihood training, but that distinction is not tested. As it stands, I am convinced that the full training recipe helps in the reported setting, but less certain which part of the probabilistic formulation is responsible.
  2. The calibration result is harder to interpret than the accuracy result because the probability convention is left unresolved. Equation (4) assigns mass to a background class. Using those probabilities directly and renormalizing over foreground classes define different NLL and Brier scores; the latter removes the dependence on the background weight entirely. Appendix B explains this distinction but does not identify which convention produced Table 5. This matters when interpreting the reported benefit of the background component. The split of the 15% label budget between representation learning and held-out concentration fitting also needs to be stated explicitly.
Questions for authors
  1. Can the authors compare the bound objective with vMF negative log likelihood on the same homogeneous backbone, using the same class-resultant update and concentration estimator? Please also clarify how concentration is updated during training and whether gradients pass through its estimate. This would help distinguish the error-bound contribution from the rest of the directional training recipe.
  2. For Table 5, are NLL and Brier computed from the full background-augmented probabilities or from renormalized foreground probabilities? How is confidence defined for ECE, and how many of the labels within the 15% budget are reserved for fitting concentration and selecting the background weight?
Reviewer_KRtGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the geometric structure of classification decision regions under positive scaling invariance and linear Bayes sufficiency. The authors show that an invariant affine readout admits a bias-free representation with polyhedral convex decision regions. Based on a shared-concentration von Mises–Fisher (vMF) model, they derive a Bhattacharyya error-bound objective and combine it with a positively homogeneous neural network. Experiments on CIFAR-100 show improvements in classification accuracy, calibration under distribution shift, and out-of-distribution detection. Ablation studies further examine the contributions of the architecture and the proposed objective.

Critical strengths
  1. The paper provides a systematic analysis of when classification decision regions have a conic structure. In particular, it distinguishes the geometric properties implied by scale invariance from those requiring an invariant affine readout, and further separates decision regions from class supports. These distinctions help clarify the assumptions under which conic decision geometry arises.

  2. The proposed method achieves improvements in classification accuracy, calibration, and OOD detection. The 2×2 factorial study is particularly useful because it separates the contribution of the positively homogeneous architecture from that of the Bhattacharyya-based objective. The results suggest that both components contribute to the observed performance improvements.

Critical weaknesses
  1. The proposed objective is motivated by the Bhattacharyya overlap between vMF class distributions. However, the main controlled comparison focuses on cosine cross-entropy. Existing approaches, such as vMF-based losses, supervised contrastive learning, and angular-margin objectives, also encourage compact within-class representations and separation between classes. Without comparisons against these closely related objectives under matched architectures and training settings, it is difficult to determine whether the proposed loss provides a meaningful advantage beyond established geometric representation learning methods. This weakens the empirical evidence for the novelty and effectiveness of the proposed objective.

  2. The main controlled experiments are conducted on CIFAR-100, and broader evaluations across datasets and architectures are missing. More importantly, the classification error bound relies on shared-concentration vMF distributions with equal class priors, while the empirical analysis indicates deviations from the shared-concentration assumption. The theoretical results therefore do not directly guarantee improved classification performance for the learned representations. Additional experiments examining the relationship between deviations from the vMF assumptions and the effectiveness of the proposed objective would strengthen the connection between theory and practice.

Questions for authors
  1. The theoretical error bound assumes equal class priors and a shared concentration parameter, while the empirical analysis indicates that class-specific concentrations differ. How sensitive is the proposed objective to violations of these assumptions? In particular, does the relationship between the Bhattacharyya objective and classification error remain meaningful when the class distributions are substantially non-vMF?

  2. The main factorial experiment is conducted on CIFAR-100. Could the authors provide evidence that the proposed objective consistently improves performance across different datasets and network architectures? This would help determine whether the observed gains reflect a general advantage of the proposed geometry or are specific to the evaluated setting.

Reviewer_gCYYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that conic decision geometry follows from positive-scaling invariance and affine Bayes optimality. The authors develop a model based on von-Mises Fisher distributions to derive a closed-form Bhattacharyya error-bound objective. Gain equivariance is enforced with a bias-free, positively homogeneous network. They demonstrate the method on CIFAR-100 and investigate calibration under shift and OOD detection.

Critical strengths
  • The theoretical statements are clean and carefully scoped. The writing is clear. The counterexample showing invariance does not imply convexity (Prop. 1) and the linearity obstruction to angular margin (Prop. 2) are both nice statements.

  • The limitations are stated candidly throughout.

Critical weaknesses
  • I might be missing something but I am not understanding the motivation of this work in the context of classification. The central idea of gain invariance is that the intended learned function is homogeneous: f(cx) = c·f(x). This can occur in a deep ReLU network without any biases, but this architecture is non-standard for classification problems. Work by Mohan et al. (2020), cited in the paper, is one example of a bias-free architecture. But the objective in that work is denoising, so the output f(x) is still in the pixel domain and it is clear why homogeneity of f is desirable (scaling the target image by c must be compensated by the denoiser scaling its output by c as well). In classification, I don't understand a clear motivation. One can, for example, always normalize the input x to unit norm before feeding it through the network rather than waiting to normalize at the final layer.

  • I hate to say it but the experiments with CIFAR-100 are a limited demonstration of the idea. There is no substantial effort to baseline the method against compute-matched or contrastive or semi-supervised baselines. Data augmentation to enforce invariance is also an obvious idea that should be baselined.

Questions for authors
  • Why not normalize the input x before feeding it through the network? Why wait to normalize at the final layer?
  • What explains the in-distribution accuracy gain from the homogeneous architecture?
  • How does the method compare with augmentation-based approaches at matched compute?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.