acceptodds
Withdrawn from ICLR 2027

Knowledge-Guided On-Policy Self Distillation for Medical MLLMs Continual Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Medical multimodal large language models (MLLMs) need continuous adaptation to emerging clinical tasks while preserving previously acquired capabilities, yet collecting high-quality trajectories and designing explicit rewards remain difficult. On-policy self distillation (OPSD) enables continual learning by supervising the model on trajectories generated via its own evolving policy without complex reward design, but the effectiveness depends on the quality of teacher-side privileged signal. In this work, we observe that OPSD-based medical MLLMs continual learning is not simply to provide more medical context, but to construct privileged knowledge conditions that yield reliable teacher supervision on uncertain student-generated trajectories. With this insight, we propose knowledge-guided on-policy self distillation for medical MLLMs continual learning (MedKgSD), a reward-free and external-teacher-free framework that uses biomedical knowledge graphs to construct training-time privileged knowledge conditions for on-policy self-distillation. For each new task, MedKgSD builds a knowledge graph induced task-level concept space and derives sample-specific privileged conditions from biomedical knowledge relations to interpret student-visited states under structured medical context. Since knowledge-derived conditions could be noisy or mismatched, we introduce token-selective entropy calibration to identify reliable teacher signals on informative student states and distills the teacher behavior along student-generated trajectories. This converts structured medical relations into dense on-policy supervision for new-task adaptation, with prior-policy distillation reducing the forgetting of previous capabilities. Experiments across multiple medical backbones demonstrate that MedKgSD improves the adaptation-retention trade-off in continual medical adaptation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multimodal Reasoning, Self-distillation, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_u3zjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an on-policy self distillation method for continue learning of medical MLLMs, MedKgSD. It uses biomedical knowledge graphs to construct privileged knowledge, and performs OPSD based on this knowledge. An entropy-based criterion is proposed to select tokens for OPSD training. Experiments on two models show that MedKgSD improves the adaptation-retention trade-off over the baselines.

Critical strengths
  • MedKgSD is a reward-free and external-teacher-free framework, which saves costs compared to RL-based methods. And self-distillation is easy to expand to other tasks.
  • Using structured knowledge graphs as the knowledge source makes sense.
  • The presentation is fair and easy to follow
Critical weaknesses
  • Limited improvement: The performance improvement of MedKgSD is not sufficient, especially on the MedVLThinker model. And all the baselines reduce performance, why? Are they suitable for the setting?
  • Two important baselines are missing: GRPO and standard OPSD. The method is based on OPSD, so standard OPSD should be included as a baseline.
  • The training and evaluation settings are not convincing. The max token is 4096, which limits the model's capabilities. For example, the QoQ model is trained under 8k token budget. Also, evaluation uses greedy decoding, which limits the model's reasoning.
  • Both QoQ-Med-7B and MedVLThinker-3B are based on Qwen2.5-VL, is it possible to provide more results on newer, stronger models, or models from other families?
Questions for authors
  • The paper 'Qoq-med: Building multimodal clinical foundation models with domain-aware grpo training' is published on Neurips 2025. Why is the reference Neurips 2026?
Reviewer_tiKrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To address the challenges of adapting to new tasks while mitigating the forgetting of previously acquired capabilities in the context of continual learning for medical multimodal large models, this paper proposes a knowledge-guided online self-distillation framework called MedKgSD. This approach leverages medical knowledge graphs to provide additional information to the backbone-based teacher model during training. It employs token-selective entropy calibration to weight teacher distributions conditioned on various knowledge inputs, and performs distillation along trajectories autonomously generated by the student model. Furthermore, it incorporates old-policy distillation to alleviate forgetting—all without the need for an external teacher or an explicit reward function. Experiments conducted across two-stage continual learning scenarios using two different medical multimodal models demonstrate that this framework achieves a superior balance between learning new tasks and retaining existing capabilities.

Critical strengths
  1. The motivation is well-justified. Medical models need to continuously learn new tasks while retaining existing capabilities; however, obtaining high-quality expert demonstrations and medical reward functions is challenging.

  2. The paper incorporates structured medical knowledge as additional information for the teacher model during training, and subsequently distills the teacher’s behavior into the student model. This approach allows medical knowledge to be translated into token-level supervision, eliminating the need for a larger external teacher model or access to a knowledge graph during the inference phase.

Critical weaknesses
  1. TSE selects and weights teachers based on entropy reduction; however, erroneous or irrelevant medical knowledge can also lead a model to become more confident. While the paper includes experiments involving the perturbation of knowledge relationships, it has not yet directly demonstrated that teachers assigned higher weights are, in fact, more accurate.

  2. The continual learning setup is too simplistic to adequately support resistance to catastrophic forgetting. The experiments consist of only two adaptation stages—"reasoning" followed by "understanding"—both derived from the same dataset.

  3. The paper compares various forms of context, such as retrieved text, entities, and graph-based relationships, but these comparisons are conducted primarily within the scope of the authors' own framework. Consequently, while the paper demonstrates the utility of structured medical knowledge, it does not sufficiently explain why this specific distillation approach is necessary.

Questions for authors

See Weakness

Reviewer_5ZnfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MedKgSD, a method for letting medical multimodal LLMs learn new clinical tasks continually while keeping their existing abilities. It is based on on-policy self-distillation: the teacher is the same model as the student, but it also sees knowledge context built from a biomedical knowledge graph (PrimeKG). A token-selective entropy weighting (TSE) decides at which positions, and how strongly, to follow each knowledge-conditioned teacher, and the student is trained toward the weighted teacher distribution on its own outputs. The authors train two medical MLLMs on a reasoning task and then an understanding task, and evaluate prior abilities on OmniMedVQA. They report a better balance between learning new tasks and keeping prior abilities than SFT, RL and other self-distillation baselines.

Critical strengths
  1. Knowledge is used only on the teacher side, so deployment has no extra cost. The knowledge graph is seen by the teacher during training only, and the trained model's input at inference time is unchanged, with no retrieval system attached. This is a practical design for medical settings.
  2. The ablations are informative. The paper compares knowledge-graph context with several alternative forms of context, which gives some evidence that the gains come from the knowledge itself rather than just from adding more text.
  3. The weighting scheme is reasonably validated, and the paper is thorough. TSE is compared with other weighting choices across multiple seeds, and the method, the knowledge-graph construction and the limitations are documented in detail.
Critical weaknesses
  1. Novelty is limited in an increasingly crowded area. The core recipe largely follows SDFT and OPCD: a same-model teacher with privileged context, on-policy distillation on the student's own outputs, and a constraint that keeps the model close to its prior policy. Selecting reliable contexts and weighting tokens is close to RoCo-ACE (arXiv 2607.24771) and Med-OPD (arXiv 2607.16303), the latter being on-policy distillation for medical vision-language models, and neither is cited. Wang et al. (arXiv 2607.01763) have already studied forgetting under on-policy self-distillation in continual post-training. The main new element is building the teacher's context from a knowledge graph, but the only experiment that isolates it is Table 3, which uses a single run and a single backbone.
  2. The paper claims continual learning and resistance to forgetting, but what it measures looks more like transfer. According to the setup in Section 4.1, the "prior tasks" on OmniMedVQA are never trained and are only used for evaluation. The gains on prior tasks in Table 1 therefore reflect transfer from training on MedXpertQA-MM, not the model retaining what it had learned. Supporting the anti-forgetting claim needs a task stream where earlier tasks are learned first and then revisited after later training.
  3. The experimental evidence is not solid enough. Tables 1–3 are single runs with no variance and no test-set sizes, while many differences are only a point or two. The baselines use neither replay nor a retention constraint, and they all share one learning rate. Under this comparison, part of MedKgSD's advantage may simply come from its extra retention mechanism.
  4. Several key details are missing from the main text. The main text does not say how the entropy threshold τ\tau is chosen, whether the teacher's parameters are frozen or updated during training, or how the "retrieved text" baseline in Table 3 is built. It is also unclear whether what gets distilled is a full reasoning trace or only the final answer. These details directly affect how the results should be read.
Questions for authors
  1. Apart from replacing demonstrations with knowledge-graph context, how does MedKgSD fundamentally differ from SDFT and OPCD? Could you discuss, and where feasible compare against, RoCo-ACE and Med-OPD, and explain how your findings relate to those of Wang et al.?
  2. Could you add an ablation that isolates the role of knowledge-graph structure? For example, compare knowledge-graph context with retrieved or plain medical text matched in concepts and length, on both backbones and over multiple seeds. How exactly was the "retrieved text" condition in Table 3 built?
  3. Could you test MedKgSD in a true continual-learning setting, training on earlier tasks, then on new tasks, and then measuring how much performance on the earlier tasks drops?
  4. Could you report mean ± std over at least three seeds for Tables 1 and 2, with per-task test-set sizes? Does MedKgSD still have an advantage if the baselines get the same replay and retention constraint?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.