From Discrete Specialists to Continuous Expertise: Continual Adaptation of Medical MLLMs via Generative LoRA

Medical multimodal large language models (MLLMs) should continually acquire capabilities across heterogeneous clinical tasks while retaining prior expertise. Existing parameter-efficient approaches face a fundamental trade-off: a single shared adapter risks interference, whereas expanding pools of task-level specialists increase storage and require expert routing. Both represent expertise at a fixed granularity, limiting scalable adaptation to individual clinical cases. We introduce C-MAGE, a framework that moves from discrete specialists toward continuous expertise by learning a shared generative rule for input-specific adaptation. Conditioned on image-instruction features, clinical department embeddings, and layer and projection identifiers, a shared generator predicts compact coefficients that combine shared basis matrices into instance-specific LoRA factors for the query and value projections of a frozen MLLM. This formulation integrates clinical priors, case-specific variation, and parameter reuse within a unified adaptation space. The framework is trained end-to-end through response prediction on current data and a small task-balanced replay memory, requiring neither pre-optimized expert checkpoints nor expert routing at inference. For evaluation, we construct MedCLAS, a clinically organized benchmark comprising 151,187 samples from 13 datasets across six departments and unifying classification, visual question answering, and report generation through a common instruction-following interface. Experiments demonstrate strong performance across heterogeneous medical tasks, showing that continual adaptation can proceed by refining shared generative expertise rather than accumulating discrete specialists. Code will be made publicly available.