acceptodds
Under review as a conference paper at ICLR 2027

Protect What Is at Risk, Distill Where It Responds: On-Policy Distillation for Continual Learning in MLLMs

Abstract

On-policy distillation (OPD) can consolidate task experts for continual learning in multimodal large language models (MLLMs), but preserving experts does not ensure effective historical supervision. Changing visual evidence, instructions, and student responses expose historical capabilities unevenly, while the general OPD ignores which capabilities are being forgotten. We argue that **historical supervision must target vulnerable knowledge on trajectories that reveal it**. Guided by this principle, we propose **R**isk-guided **E**xpert **S**election via functional **P**robes for **ON**-policy **D**istillation (**RESPOND**), the first continual OPD framework to couple *what knowledge needs protection* with *where it can be distilled*. RESPOND probes current trajectories to supervise capabilities that are vulnerable in the student and observable through their experts, enabling consolidation with bounded historical-teacher queries per update. Experiments on eight sequential tasks and two OOD benchmarks show RESPOND improves final overall averages by 2.25–2.91 points over the strongest competing continual methods across three backbones. On Qwen3.5-4B, RESPOND reduces average forgetting by 50.2% and 69.7% relative to two vanilla continual OPD baselines.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Multimodal Large Language Model, On-Policy Distillation, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_e6VTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RESPOND, a continual on-policy distillation framework that consolidates independently trained LoRA task experts into a single student. It uses forgetting-risk estimates to select historical experts and LoRA response probes to identify informative current-task trajectories for distillation. Experiments across three MLLM backbones report improved accuracy and retention on an eight-task stream.

Critical strengths
  1. Jointly selecting historical experts according to forgetting risk and trajectory-level response addresses a meaningful challenge in continual OPD.

  2. Experiments across three backbones, component ablations, and proxy-correlation analyses provide encouraging evidence for the proposed selection mechanisms.

Critical weaknesses
  1. It remains unclear whether supervision on current-task trajectories adequately protects historical capabilities. The selected probes may have limited coverage, especially for short responses or mismatched tasks, while stage-wise risk resets may overlook previously degraded capabilities.

  2. The practical CL advantage is not fully established. Independent expert training, a growing expert library, and repeated probing introduce unreported costs. The custom evaluation stream and large performance degradation of some baselines also warrant further validation.

Questions for authors
  1. Since historical experts are distilled only on current-task student trajectories, what evidence shows that risk- and response-guided selection effectively preserves historical capabilities, beyond capturing high teacher KL signals? How does the method behave when current and historical tasks have little overlap?

  2. How are capabilities already degraded before a new stage protected, given that the risk reference and persistent states are reset?

  3. What is the total computational cost of RESPOND, including expert training, probing, and consolidation, compared with compute-matched CL baselines? As the number of tasks grows, how do expert storage, probing, and selection costs scale, despite the limit of three historical-teacher queries per update?

  4. Could the authors explain the substantial performance differences of SDPO across backbones, particularly on COCO (25.22 with Qwen3.5-4B versus 74.89 with Qwen3-VL-4B), and clarify whether the baseline configurations were appropriately tuned for each model?

Reviewer_pDKrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

RESPOND studies continual consolidation of task-specific multimodal experts into a single student without replaying historical inputs. It combines retrospective drift and prospective interference estimates with adapter-response probes to select historical teachers and informative current trajectories for on-policy distillation. Experiments across three backbones and an eight-task stream report improved final performance and reduced forgetting.

Critical strengths
  1. The distinction between protection demand and available supervision is well motivated. Coupling these two considerations provides a coherent approach to consolidating a growing expert library into a task-agnostic student.
  2. The empirical support is substantial: aggregate gains across backbones, stage-wise retention results, component ablations, and proxy diagnostics jointly support the method's practical value. The study goes beyond reporting final benchmark improvements alone.
Critical weaknesses
  1. Attribution relative to uniform OPD. Simple-OPD and RESPOND use different current/historical loss weightings (Eqs. 3 and 18). Table 3 provides useful component evidence, but the replacement policies and query budgets of its ablations are insufficiently specified. The contribution of selective allocation, beyond rebalancing supervision, is therefore not fully isolated.
  2. Connection to historical capability. The proxy diagnostics establish meaningful associations with gradient conflict and teacher disagreement on current-input probes. Their connection to damage on historical tasks remains indirect; the coverage condition in Section 4.3 is an assumption. This limits the strength of the proposed mechanism's interpretation, despite the positive retention results.
  3. Practical cost accounting. The cap on full historical-teacher queries is clear, but response scoring still involves the historical expert library, and calibration adds queries. Without measured overhead, the practical trade-off introduced by this selection machinery is difficult to assess. This is a secondary concern.
Questions for authors
  1. Please specify the replacement strategies in Table 3 and whether loss weights, query budgets, and update counts are matched. If they are not, could one representative matched uniform-selection comparison isolate the allocation benefit?
  2. Using an existing checkpoint, does the risk ranking predict historical held-out loss increases under a hypothetical current-task-only update? A focused diagnostic, or clarification of existing evidence, would strengthen the proposed link.
  3. Could existing-run measurements report average historical-teacher queries and consolidation time, including response scoring and calibration, relative to Simple-OPD?
Reviewer_BJenOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Summary: This paper proposes RESPOND, a risk-guided on-policy distillation framework for mitigating catastrophic forgetting in continual learning for multimodal large language models (MLLMs). The key idea is to selectively distill historical knowledge based on both forgetting risk and the usefulness of current training trajectories. Experiments across multiple MLLM backbones demonstrate improved overall performance and reduced forgetting compared with existing continual learning methods.

Critical strengths

Strengths: (1) The joint consideration of forgetting risk and distillation opportunity is well-motivated and addresses limitations of uniform expert supervision. (2) The lightweight LoRA-based functional probing enables efficient expert and sample selection, supported by strong experimental results and ablation studies.

Critical weaknesses

Weakness: The method relies on current-task samples to activate historical capabilities, which may be less effective when tasks have substantially different distributions. More evaluation under such challenging scenarios would strengthen the paper. Nevertheless, this limitation does not undermine the paper's main contribution or its overall effectiveness.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.