WRSS: Severity Transport for Hierarchical Ranking Evaluation

Hierarchical classification errors differ not only in correctness, but also in severity. For ranked predictions, evaluation must additionally account for where severe alternatives appear. We study this problem as severity displacement: the movement of hierarchy-derived severity toward the head of a predicted ranking. We introduce the Wasserstein Rank–Severity Score (WRSS), which measures this displacement through one-dimensional transport over rank positions. The resulting distance is equivalent to a linear rank-weighted severity objective, while the transport construction jointly determines its weighting, normalization, equal-tier invariance, and mass-preserving top-$k$ truncation. Controlled experiments separately test stability under stochastic ranking ambiguity and responsiveness to hierarchy-derived severity corruption. WRSS exhibits lower local volatility than HOPS in the tested ambiguity regimes and stronger alignment with controlled severity displacement in several fixed-budget settings. Experiments on trained classifiers show similar stability under input perturbations. WRSS therefore complements distance-based and preference-based metrics by measuring cumulative severity displacement along the ranking.