TopoTuner: Topological Finetuning of Large Language Models

Full fine-tuning makes all pretrained parameters trainable and can be computationally expensive. LoRA reduces the number of trainable parameters, but it does not directly answer which pretrained components should be trained and which can be frozen during adaptation. We introduce TopoTuner, a topology-guided fine-tuning framework for selective freezing of attention projection matrices. TopoTuner treats each projection matrix as a row cloud and uses Wasserstein distances between persistence diagrams to measure how its topology changes during fine-tuning. TopoTuner learns a reusable freezing profile from a source dataset and reuses the selected matrix indices to efficiently fine-tune models on unseen target datasets, both within and outside the source domain. We evaluate whether the same freezing decisions remain effective across question answering and sentiment analysis tasks. Across LLaMA-3.1-8B, Mistral-7B-v0.3, and Qwen3-8B-Base, TopoTuner is competitive with full fine-tuning while updating only 0.56%–4.85% of pretrained parameters and outperforms LoRA in 8 out of 9 model-dataset settings. Along with minimized updates, TopoTuner reduces training time by 16.5% relative to full fine-tuning and 5.3% relative to LoRA on average. TopoTuner opens a new direction for Efficient AI where matrix-freezing decisions learned on one dataset can be reused across multiple tasks.