- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes C-MAGE, a framework for continual adaptation of medical multimodal language models. A shared hypernetwork generates instance-specific LoRA factors through shared basis matrices, conditioned on image-instruction features and department metadata. The authors also introduce MedCLAS, which organizes 13 datasets across six departments into a continual-learning benchmark. Experiments compare C-MAGE with continual-learning baselines and examine its components, task orders, replay budgets, and backbones.
Critical strengths
-
Useful benchmark organization. MedCLAS brings classification, visual question answering, and report generation into a common continual-learning setting across medical specialties, enabling evaluation of heterogeneous capability acquisition and retention.
-
Concrete evidence for the proposed parameterization. The shared-basis formulation substantially outperforms direct LoRA-factor generation in the reported ablation. Additional evaluations across task orders, replay budgets, and backbones help assess its applicability beyond a single configuration.
Critical weaknesses
The methodological novelty is not sufficiently established. HyLoVQA[1] already applies sample-conditioned LoRA generation to continual VQA, Text-to-LoRA[2] supports training the generator directly through task supervision, and HyperLoRA[3] explores low-dimensional generation using basis matrices and predicted coefficients. Although the proposed factorization with bases shared across layers has concrete differences, the paper lacks sufficient discussion and comparison with these closely related approaches to establish its distinct methodological contribution.
[1] Wang, Y., Xiong, C., Qin, Z., Zhang, M., Xiao, K., & Li, Z. (2026). HyLoVQA: Dynamic Hypernetwork-Generated Low-Rank Adaptation for Continual Visual Question Answering. arXiv preprint arXiv:2605.22035.
[2] Charakorn, R., Cetin, E., Tang, Y., & Lange, R. T. (2025). Text-to-LoRA: Instant transformer adaption. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 7485–7514). Proceedings of Machine Learning Research.
[3] Li, M., Chen, J., Feng, W., Li, B., Dai, F., Zhao, S., & He, Q. (2025, June). Hyperlora: Parameter-efficient adaptive generation for portrait synthesis. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 13114-13123). IEEE.
Questions for authors
-
Could authors clarify the methodological differences from HyLoVQA, Text-to-LoRA, and HyperLoRA, and explain which limitations of these approaches are addressed by sharing factorization bases across layers?
-
Why do shared basis matrices yield such a substantial improvement? Is there further evidence distinguishing easier parameter generation from improved knowledge sharing?
-
Do all baselines receive the same department information and use comparable training budgets? Were the main experiments repeated independently, particularly to establish whether the 1.06-point advantage over Replay without department conditioning is stable?
7 comments
Response#2-HTGw
Concern 2: Why do shared basis matrices yield such a substantial improvement? Is there further evidence distinguishing easier parameter generation from improved knowledge sharing?
Response: Shared bases jointly simplify parameter generation and introduce explicit sharing among generated adaptations. The generator predicts low-dimensional coordinates within trainable factor subspaces, rather than every LoRA factor entry. Under our configuration, this requires 4,096 coefficients per adaptation site instead of 131,072 query-factor entries or 81,920 value-factor entries, revealing a 32-fold and 20-fold reduction, respectively. The bases also provide adaptation-subspace sharing beyond sharing the hypernetwork itself. The same projection-specific bases receive learning signals from different inputs, transformer layers, and successive tasks. Case- and layer-conditioned coefficients preserve distinct adaptations within these common subspaces. This makes parameter reuse explicit while retaining specialization for individual clinical cases.
The following comparisons provide complementary evidence:
| Configuration | Final Avg. Score ↑ |
|---|---|
| Direct factor generation | 34.75 |
| Shared-basis generation without department conditioning | 65.87 |
| Department-only conditioning | 62.80 |
| Full C-MAGE | 68.16 |
With image-instruction conditioning retained and department conditioning absent, shared-basis generation improves performance by 31.12 points over direct generation. The department-only comparison further holds the generator, shared bases, coefficient dimensionality, and training protocol fixed. Introducing case-level conditioning improves the score by 5.36 points and BWT from −8.65 to −6.34. Thus, compact generation alone does not reproduce the full model’s performance; case-adaptive use of the shared space contributes additional gains. These results establish the effectiveness of the structured parameterization and the value of case-specific conditioning.
Concern 3: Do baselines have the same department information and comparable training budgets? Is the improvement without department conditioning stable across independent runs?
Response: The reported results are averages over three independent runs, and our comparisons use a common backbone, dataset splits, task sequence, evaluation protocol, and dataset-level training budget. Department conditioning is an explicit component of our hospital-oriented method, and its contribution is separately evaluated. Hospital-wide continual adaptation of medical MLLMs across heterogeneous specialties, modalities, and task types remains comparatively underexplored. This motivates exploiting the clinical structure available in hospital workflows: department context supplies a coarse service-level prior, while image-instruction features specify case-level adaptation requirements.
Thank you for the detailed responses and additional experiments, which address most of my concerns.
I now lean toward weak accept based on the clarified contribution, the additional ablation, and the benchmark contribution. However, I remain uncertain about the very large performance gap in the newly added HyLoVQA comparison, as the adaptation and implementation details are insufficient to assess its reliability.
I am therefore raising my rating while lowering my confidence to reflect this remaining uncertainty.
Response#1-HTGw
We sincerely thank Reviewer HTGw for the constructive assessment. We appreciate your recognition of MedCLAS, the empirical support for shared-basis parameterization, and the additional sensitivity analyses. Regarding your concerns, we respond point by point below.
Response: Thank you for highlighting these approaches. C-MAGE’s methodological contribution is a clinically conditioned adaptation space that explicitly shares factor subspaces across layers while preserving case-specific specialization. This structure supports continual refinement across heterogeneous hospital services without accumulating dataset-specific adapters. The distinction lies in how adaptations are represented, conditioned, and continually shared—not simply in using a hypernetwork.
Regarding the closest relevant one, HyLoVQA generates LoRA adapters from retrieved visual-object and textual-task anchors, with semantic-functional alignment for continual VQA. C-MAGE instead combines multimodal case features and department priors to generate adaptations within cross-layer shared factor subspaces. We additionally evaluate HyLoVQA on MedCLAS under the same experimental protocol:
C-MAGE improves the final Avg. Score by 40.42 points and BWT by 2.47 points. Even without department conditioning, C-MAGE achieves 65.87, showing that its advantage persists without the additional clinical prior. These results provide direct evidence beyond a qualitative distinction between architectures.
Regarding Text-to-LoRA, it supports adapter reconstruction, direct task supervision, and layer/module conditioning; we do not claim these mechanisms as new. Its original formulation learns task-description-conditioned adapters. C-MAGE learns a continually updated mapping from individual medical images, instructions, and department context to adaptations within shared factor subspaces. Consequently, cases within the same service can receive different adaptations according to their visual evidence and response requirements.
For HyperLoRA, it predicts coefficients over trainable LoRA matrix bases for portrait synthesis. Beyond the totally different task, C-MAGE instead introduces projection-specific factor bases shared across transformer layers, with coefficient matrices generated for each clinical case and adaptation site. Both the generator and bases are refined through sequential medical task supervision.
What cross-layer sharing addresses. Maintaining separate bases for every layer duplicates basis parameters and provides no explicit coupling through those bases. C-MAGE replaces these separate parameter sets with common trainable factor subspaces. Learning signals from different layers and successive tasks therefore update the same bases, while layer-conditioned coefficients preserve distinct adaptations at each site. For our 32-layer backbone, this reduces basis parameters 32-fold relative to a layer-specific implementation of the same factorization. The empirical evidence supports the value of structured generation: with image–instruction conditioning retained and department conditioning absent, shared-basis generation improves the final Avg. Score from 34.75 to 65.87. Adding the clinical prior further increases it to 68.16. This ablation evaluates the shared-basis formulation as a whole; the cross-layer storage reduction follows directly from its architecture. Together, these distinctions establish C-MAGE as a structured approach to continual medical adaptation, with explicit mechanisms for cross-layer parameter reuse, case-level specialization, and service-level clinical conditioning.