- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GaussianFC and GaussianFC-PDF, which parameterize each feature’s weight profile along the label axis with a Gaussian-shaped function to exploit label order. The authors analyze their effects through theoretical analysis and head-swap experiments, and compare them against eight baselines across 23 experimental configurations covering four types of tasks (Sections 4 to 6).
Critical strengths
The method is simple, the experiments cover diverse settings, and hyperparameters are selected on a nested validation split. The head-swap experiments also support that some representation benefits carry over to other readouts. On FG-NET with ViT-Tiny, training a linear head on a frozen GFC-trained backbone achieves an MAE of 4.87, compared with 7.17 for end-to-end Linear+CE training. Similar improvements appear on both ResNet backbones (Figs. 2 and 5).
Critical weaknesses
W1: The additional value of the Gaussian parameterization is not sufficiently established.
coGOL (Lu et al., SDM 2022) already incorporates label order by penalizing weight differences between adjacent threshold classifiers. The semicircular cosine variant of Hard-CPL (Wang et al., AAAI 2023) constructs class proxies in label order and uses scaled cosine similarity to compute logits. These approaches differ from the proposed per-feature Gaussian parameterization, but show that the abstract’s characterization of prior work is too broad.
The experiments lack direct comparisons with heads that regularize classifier-weight smoothness or impose ordered prototype structures. The current evidence therefore does not sufficiently establish the additional benefit of choosing a Gaussian profile over other ways of imposing ordinal structure on the head.
W2: The interpretation of ordered representations goes beyond the formal theoretical results.
Proposition 3 bounds the component of a one-step update outside a particular feature subspace; it does not guarantee that class centers follow label order. Eq. (21) also contains a term reflecting how class-conditional mean predictions vary along the label axis. The current analysis therefore does not establish that head smoothness alone guarantees smoothness of the complete gradient field.
The authors acknowledge relevant limitations in Appendix A.5 and Section 7. However, the abstract presents an implicit bias that “drives the backbone toward ordered feature geometry” as a proven mechanism. Appendix B first bounds the target-driven term −W, but it is unclear whether its subsequent claim of smoothness “at every step, for every residual field” is restricted to that term. The scope of these claims should be made explicit and consistent with the formal results.
Questions for authors
[Clarification] Which existing results support an additional benefit of Gaussian parameterization over other ways of imposing ordinal structure on classifier heads?
[Clarification] Is the smoothness claim in Appendix B restricted to the target-driven term, or does it concern the complete gradient field? How do Proposition 3 and Eq. (21) support the mechanism claimed in the abstract, and which parts currently rely only on empirical evidence?