AMID: Attribution Matching and Inverse-Network Distillation for Heterogeneous Super-Resolution

Distilling image super-resolution (SR) models across heterogeneous architectures is challenging because their hidden representations are not directly comparable, while output matching alone does not transfer how the teacher exploits low-resolution evidence. We propose Attribution Matching and Inverse-Network Distillation (AMID), which uses teacher outputs and output gradients without accessing teacher intermediate features. Global Attribution Map Distillation (GAMD) matches integrated-gradient attribution maps of guidance-weighted structural responses, transferring the teacher's low-resolution pixel utilization and stabilizing reconstruction fidelity. Inverse Self-Consistency Distillation (ISCD) passes teacher and student outputs through a frozen inverse network architecturally isomorphic to the student, enabling multi-layer contrastive alignment in a shared, degradation-aware proxy space. An inverse reconstruction constraint further enforces degradation consistency. Experiments with five heterogeneous teachers across five benchmarks demonstrate improvements over representative distillation methods at $2\times$, $3\times$, and $4\times$. At $3\times$ with a SwinIR teacher, AMID outperforms all compared baselines across all eight metrics and improves MUSIQ by $1.22$ over the strongest competing method. All auxiliary components are used only during training. AMID preserves the student's architecture and inference cost, making it practical for deployment-oriented heterogeneous SR distillation.