BRIDGE: Bridging Expert and Learner Trajectories for Black-box LLM Agent Distillation

Distilling capable large language model (LLM) agents into smaller models offers a practical path toward efficient deployment, yet existing approaches typically specialize in either imitation from expert trajectories or correction on student-generated trajectories, leaving the two complementary supervision regimes disconnected. More fundamentally, both paradigms often couple trajectory provenance with supervision direction, treating teacher-generated behavior as the learning target regardless of its realized quality. We introduce BRIDGE, a black-box agent distillation framework that bridges expert and learner trajectories within a unified preference-learning framework while decoupling trajectory source from preference direction. BRIDGE constructs comparisons from both expert- and learner-induced histories and determines their labels from realized trajectory quality, allowing either teacher- or student-generated behavior to provide the positive learning signal. It further explores informative recurrent states and terminates local comparisons upon environment-state reconvergence, yielding finer-grained credit assignment while limiting unnecessary policy divergence. BRIDGE requires only teacher-generated text and environment feedback, without access to teacher logits or internal representations. Across WebShop, ALFWorld, and ScienceWorld, BRIDGE consistently outperforms strong imitation- and correction-based distillation baselines, and remains effective with DeepSeek-R1-Distill-Llama-8B students, demonstrating robust black-box distillation across heterogeneous model families. The anonymous code repository is available at https://anonymous.4open.science/r/bridge123/.