Heterogeneous Socialized Learning: Enabling Individual Growth Across Diverse Model Architectures

Collaborative learning among heterogeneous agents offers a promising approach to overcoming the limited knowledge of individual agents. However, existing heterogeneous federated learning methods primarily focus on improving global or local task performance, rather than enabling each agent to acquire complementary knowledge beyond its original expertise. Although socialized learning emphasizes individual knowledge acquisition and preservation, its reliance on shared model architectures limits its applicability to heterogeneous agents. To address these challenges, we formulate Heterogeneous Socialized Learning, a new paradigm that enables agents with distinct model architectures to acquire complementary knowledge from their peers while preserving their original expertise. We further propose Heterogeneous Socialized Collaboration (HSC), an architecture-agnostic framework that facilitates cross-model knowledge exchange through compact knowledge packets constructed via dataset distillation. By integrating peer knowledge distillation with feature retention, HSC enables each agent to acquire new knowledge while retaining its existing expertise and model architecture. Extensive experiments on CIFAR-100 and Tiny ImageNet demonstrate that HSC consistently improves new knowledge acquisition and overall competence across diverse model architectures and agent populations while retaining substantial original expertise.