Null-Space Orthogonal Fine-Tuning for Multimodal Continual Instruction Tuning

Multimodal Continual Instruction Tuning (MCIT) requires Multimodal Large Language Models (MLLMs) to adapt to new tasks while retaining previously learned knowledge, making the balance between stability and plasticity a central challenge. Orthogonal Fine-Tuning (OFT), traditionally applied to parameter-efficient downstream adaptation, offers a potential starting point for MCIT through orthogonal transformations. In MCIT, orthogonality alone does not explicitly control interference with previous tasks. A null-space constraint can reduce such interference, but simply imposing it on OFT is insufficient to guarantee orthogonality. Therefore, we propose Null-space Orthogonal Fine-Tuning (NOFT), which parameterizes task-specific transformations to jointly satisfy the null-space and orthogonality constraints, thereby improving stability on continual adaptation. To enhance plasticity under OFT's fixed block-diagonal structure, NOFT further introduces plasticity-aware permutation, which rearranges the trainable block positions to capture more current-task gradient information without increasing the parameter budget. Experiments with LLaVA-v1.5-7B and Qwen3-VL-8B on two MCIT benchmarks demonstrate that NOFT effectively balances stability and plasticity and achieves state-of-the-art performance.