Beyond Standalone Accuracy: Class-Frequency Correspondence Shapes Conditional Expert Gain in Long-Tailed Recognition

Evaluating an additional expert in multi-expert long-tailed recognition requires accounting for the correctness already covered by the existing expert set. Standalone accuracy cannot distinguish whether an auxiliary expert repeats existing coverage or recovers residual errors. We therefore define conditional expert gain to measure correctness contributed beyond existing coverage and establish a set-conditioned view of complementary expert value. Using this view, we identify class-frequency correspondence as a key factor shaping residual specialization, the distribution of newly recovered correctness across the class-frequency spectrum. Controlled correspondence interventions reveal an accuracy-gain inversion: reversing the class-to-frequency assignment raises standalone accuracy while sharply reducing conditional gain and shifting recovery away from rare classes. Class-accuracy-preserving analysis further shows that this gain shift is not explained by class-wise accuracy allocation alone, revealing an additional within-class residual-overlap contribution. Guided by this insight, we propose Frequency-Structured Auxiliary Learning (FSAL), which uses a shared correspondence to orient frequency-dependent auxiliary scores within the inherited two-stage structure and expert budget, enforcing one class-wise frequency assignment across the auxiliary objective. On CIFAR-100-LT, FSAL achieves 59.5% Overall Top-1 accuracy and 44.1% Few-class accuracy at IR=100, with the highest Overall Top-1 among the compared methods at all three imbalance ratios. The same correspondence-dependent recovery shift extends to ImageNet-LT, while results on iNaturalist 2018 further support transfer under natural class imbalance. Code is available at Supplement.