FedSocratic: Exploring Knowledge Transfer through Socratic Questioning in Heterogeneous Federated LLMs

Federated learning enables collaborative improvement of large language models while keeping training data local. However, architectural differences hinder parameter aggregation, while output-based knowledge exchange often relies on shared public training corpora. We study how heterogeneous LLMs can identify and transfer complementary knowledge without a shared public training corpus or parameter alignment. Inspired by Socratic inquiry, we propose FedSocratic, which combines structured questioning, answer verification, and local adaptation. Base questions and answer-preserving and answer-changing follow-ups probe client capabilities; executable rules verify final answers, and receivers learn from accepted supervision while retaining their architectures. Experiments with four heterogeneous clients across four verifiable task domains demonstrate improved performance on unseen procedural states. Controlled studies show that complete question coverage improves pooled weak-domain accuracy over omitting question types at matched record and update budgets. With response text retained, capability-selected peers outperform verified self-training and random peers in mean cross-domain accuracy on matched accepted questions and record counts. These findings support knowledge transfer through structured questioning within verifiable task families and highlight question coverage and supervision-source selection as key design choices.