TangentFlow: Spectral Tangent Distillation for One-Step Generative Models

One-step generative distillation trains a student to replace a multi-step teacher with a single network evaluation. We study whether explicitly matching selected teacher Jacobian responses can improve the student's geometric fidelity while maintaining image quality. We introduce TangentFlow, a spectral tangent objective that caches the leading singular directions of the teacher's noise-to-image Jacobian and trains the student to match the corresponding responses with spectral weighting. Across CIFAR-10 and ImageNet-64, TangentFlow achieves the lowest geometric error among the evaluated methods on all six measures, including a 40% reduction in retained-subspace response error on held-out CIFAR-10 latents relative to the matched distribution-matching and adversarial baseline (three seeds) and an 8% reduction on held-out ImageNet-64 latents. It reduces Fréchet Inception Distance (FID) by 21% relative to the matched CIFAR-10 baseline and by 3.8% relative to distribution matching on ImageNet-64. Our analysis motivates direction selection through teacher-response energy retention, establishes relative-error bounds on the selected subspace, shows that the whitened loss equals the average relative response error at every error level, and shows that cached-target error makes whitening reliable only on leading directions, which explains the observed interaction between direction selection and weighting. These results support spectral tangent supervision as a way to improve geometric fidelity while maintaining image quality.