Beyond Surrogates: A Quantitative Analysis for Ranking Metric Relationships

The **Consistency** property between surrogate losses and evaluation metrics has been extensively studied to ensure that approaching the optimal surrogate risk leads to metric optimality. However, such guarantees do not ensure that different evaluation metrics improve together during optimization. This gap limits our understanding of the “Metric Mismatch” observed when improvements in one metric fail to translate into gains in another. To bridge this gap, this paper proposes a unified theoretical framework designed to quantify the relationships between ranking metrics. For fixed binary-relevance lists, we characterize the optimal linear **Regret Transfer** constants between ranking metrics and show that global regret transfer and local gain sensitivity can exhibit different scaling behavior. We further investigate **Dynamic Gain Transfer** through changes in cumulative relevance across ranking prefixes, deriving exact metric-gain identities and conditions for simultaneous improvement or conflict. To connect these relationships to optimization, we analyze score-margin dynamics and prove that equal-weight population binary cross-entropy gradient flow with independently optimized logits introduces no new relevance-order inversions. Through this framework, we provide a new perspective on how the location and direction of ranking changes govern cross-metric improvements, offering a theoretical basis for understanding metric alignment along optimization trajectories.