acceptodds
Under review as a conference paper at ICLR 2027

Recursive Continual Post-Training of Foundation Models

Abstract

Foundation models should be able to learn new tasks after deployment without forgetting old ones or becoming larger over time. Current approaches tend to preserve one at the expense of the other: self-distillation keeps the model size fixed, but is limited by the model’s own capabilities; expert routing protects past tasks with dedicated frozen experts, but every new task makes the deployed model larger and more expensive to run. We present Recursive Continual Post-Training (ReCap), which captures the benefits of frozen experts without deploying them. At each stage, ReCap trains an expert on the new task, then distills it back into a single base-sized checkpoint. For prior tasks, a small per-task generator reconstructs input embeddings, which the corresponding frozen expert relabels to recover its behavior in the student. These frozen experts provide stable distillation targets across later stages. On CoIN and UCIT-O, ReCap reaches and average accuracy, respectively, matching or outperforming the strongest expert-routing baselines. It also improves average performance across six held-out benchmarks by 2.1 points over the base model, with no benchmark dropping. Similar results hold across Qwen and Llama models from 1B to 7B, and across both vision-language and text-only task sequences.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continuous Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GCANOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Recursive Continual Post-Training (ReCaP), a framework that combines the benefits of distillation and expert expansion in continual learning (CL). Specifically, it learns and stores a LoRA expert per-task, then performs multi-teacher distillation to distill the experts' knowledge into a shared base model. Extensive experiments show that ReCaP outperforms SOTA CL models on multiple benchmarks and backbones.

Critical strengths
  1. The proposed method in the paper is well-motivated, combining the advantages of two widely used strategies to resolve CL issues.

  2. The paper conducts thorough experiments on multiple backbones and benchmarks. Most results support the superiority of ReCaP. The influences of design choices in the model are well discussed.

Critical weaknesses
  1. Although the inference cost remains O(1)O(1), the training of ReCaP requires storing experts and flow-matching models for tasks, and the distillation takes extra time. What is the computational overhead introduced in this process? Specifically, how does the stored memory grow with tasks? What is the overall time consumption in training and inference, compared to other models (e.g., MoE models)?

  2. In Table 3, ReCaP does not outperform previous models in LLM 3-stage stream. It could be helpful if the authors can provide more insights to explain this phenomenon.

Questions for authors

Please see the weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer GCAN: Training and Inference Cost, Text-Only Stream

We thank the reviewer for finding ReCaP well-motivated in combining the strengths of distillation and expert expansion, its experiments thorough across multiple backbones and benchmarks with most results supporting its superiority, and its design choices well discussed.

W1: Although the inference cost remains […], the training of ReCaP requires storing experts and flow-matching models for tasks, and the distillation takes extra time. What is the computational overhead introduced in this process? Specifically, how does the stored memory grow with tasks? What is the overall time consumption in training and inference, compared to other models (e.g., MoE models)?

A1: Training storage grows by under 6% of the base per task, training time by 21% over expert training on CoIN, and inference is cheaper than expert pools.

Inference (one A100, without any optimization). ReCaP deploys one merged checkpoint with the base architecture:

MethodPeak GPU memoryLatency, batch 1Throughput, batch 8
ReCaP7.1 GiB2.3 s2.84 req/s
LoRAMoE10.1 GiB5.0 s0.33 req/s
HiDe10.1 GiB4.5 s0.31 req/s
SOCiaL9.2 GiB4.6 s0.52 req/s

Expert pools keep their experts and routers in memory; ReCaP's serving cost is the base model's and does not grow with the number of tasks.

Storage. All experts share one frozen base, so for T tasks the training archive is Marchive=Mbase+T(Mexpert+Mflow)M_{\text{archive}}=M_{\text{base}}+T(M_{\text{expert}}+M_{\text{flow}}):

ItemQwen2.5-VL-3BQwen2.5-7B (text-only)
Shared base checkpoint7.5 GB15.2 GB
Expert adapter, per task223 MB646 MB
Flow memory, per taskAbout 200 MB206 MB
Added storage per taskAbout 423 MB852 MB

The archive grows linearly with T, which the revision states as a limitation, but each task adds under 6% of the base and the archive is never deployed. Expert pools instead keep their experts at deployment.

Training time (CoIN, reported runs):

PhaseWall timeGPU-hours
Eight task experts6.3 h on 8 A100s50.5
Flow memory (ImageNet)70 min on 1 A1001.2
Consolidation, eight stages of 500 steps72 min on 8 A100s9.6
Added by ReCaP--10.8 (21%)

Expert training is shared with expert-pool methods, which also train one adapter per task; ReCaP's addition is paid once, while its inference savings apply to every request. On the text-only stream, each flow takes about 30 min on one A100 and the three consolidation stages take 52, 73 and 75 min on two A100s.

W2: In Table 3, ReCaP does not outperform previous models in LLM 3-stage stream. It could be helpful if the authors can provide more insights to explain this phenomenon.

**A2:The text-only stream is short and small (three tasks, 14,098 training samples); EWC is best on it and ReCaP is second, and the gap is the last task.

Science (5,000 samples)Tool (4,098)Medical, last (5,000)Skill avg.Retention avg.
Base71.152.640.754.869.7
Task experts (ReCaP's teachers)78.472.748.466.5--
EWC76.659.247.761.267.0
ReCaP (Table 3)76.061.841.659.866.4
ReCaP, medical stage 3,000 steps (new)77.263.544.361.665.8

ReCaP is level with EWC on science and ahead on tool; the gap is medical, and it is answer length rather than medical knowledge. Medical is the only long-form task (median reference 88 words), scored by token F1, while science and tool, learned first, have short answers (a letter, a JSON call). ReCaP's medical answers stay short (median 62 words), so they lose recall; at matched length its F1 equals the expert's (51.5 vs 52.7). Consolidating medical alone from the base model with the same recipe reaches the expert (48.2 vs 48.4). Training the medical stage longer (3,000 instead of 1,000 steps) raises medical to 44.3 while keeping tool use, and ReCaP's skill average (61.6) then exceeds EWC's (61.2).

StreamEWCReCaP
CoIN, 8 tasks67.578.0
UCIT, 6 tasks57.781.0
UCIT-O Protocol 1, 10 tasks61.175.5

On the longer vision-language streams the ranking reverses, so EWC's advantage is specific to this short text-only stream.

We hope the measured storage, training and inference costs, and the analysis of the text-only stream, address the reviewer's concerns, and would appreciate the reviewer considering an increased score.

Reviewer_WntrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes ReCaP, a framework for continual post-training in which a single model acquires tasks one at a time through recursive, stage-wise updates. At each stage, a task-specific adapter over a frozen base is trained and kept as a frozen expert, while a flow matching model captures the task's input-embedding statistics. Afterwards, the task specific experts are consolidated into a single shared checkpoint through distillation. In traditional regimes, learning a new task requires retraining or fine tuning an expert or a model on the new tasks and this runs the risk of the model forgetting the earlier tasks. This makes learning successive tasks difficult. ReCaP’s goal is to solve this problem while also achieving a single fixed sized model checkpoint without needing task specific routers.

Critical strengths

One of the core strengths and at the same time, one of the core components of the paper is the generative replay of the embedding space and it is ablated. The authors show the effectiveness of the generator model in the setup with task specific evaluations.

ReCaP performs similar or almost identical to their more expensive and forgetting prone counterpart methods, e.g. multi-task training. Through ReCaP, a model can be trained recursively on newer tasks and data without running the risk of losing prior task capabilities.

Since the method generates a single fixed size checkpoint, it makes deployment and evaluation comparatively easier.

Critical weaknesses

During replay coherence, a set of prior experts are retained and not all of them. The motivation behind this selection process wasn’t clarified in the paper. Also, the ablations for expert selection, e.g. if all the experts were selected are also absent.

ReCaP’s benefits are grounded in being routerless and ease of deployment, since there is only one resulting checkpoint. There is no comparison with other methods or counterparts for deployment in terms of compute and memory. To be more specific, if one were to ask why this framework is beneficial for deployment, regardless of the number of checkpoints or experts, there is no clear answer.

Questions for authors

Can you please explain what P_k is in section 3.2 ?

In the self-distillation comparison, can both methods be run under both task orders? Without order matching the current table supports no conclusion about self-distillation versus frozen-expert distillation.

How was the joint multi-task reference model trained? (please refer to table 3, LLM 3-stage stream comparisons)

Also from table 3, if a jointly trained multi-task model performs better than ReCaP trained experts, can you make a case for your method where a jointly trained model would be a more reasonable alternative over ReCaP?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Wntr (1/2): Expert Selection, Measured Forgetting, and Deployment Cost

We thank the reviewer for recognizing ReCaP's strengths: generative replay in the embedding space is its core component and is ablated; ReCaP performs close to the more expensive multi-task training while learning new tasks recursively without losing earlier skills; and its single fixed-size checkpoint makes deployment and evaluation easier.

W1: During replay coherence, a set of prior experts are retained and not all of them. The motivation behind this selection process wasn’t clarified in the paper. Also, the ablations for expert selection, e.g. if all the experts were selected are also absent.

A1: Algorithm 1 has no selection step: ReCaP keeps every expert and replays them in round-robin order. The reported runs, however, replayed a hand-chosen subset (ImageNet on CoIN), and we thank the reviewer for pointing out that this choice was not motivated. Prompted by this question, we added a rule that measures forgetting and selects the experts to replay.

Selecting by measured forgetting. Let ϵt(k)\epsilon_t^{(k)} be the replay KL of Eq. (4) from the frozen expert EkE_k to the student StS_t on inputs generated by FkF_k. Its rise since task k was learned measures forgetting without stored data or labels, and a task is replayed once the rise exceeds τ = 1:

Δt(k)=ϵt(k)−ϵk(k),Rt+1=Rt∪{k<t: Δt(k)>τ}.\Delta_t^{(k)}=\epsilon_t^{(k)}-\epsilon_k^{(k)},\qquad \mathcal R_{t+1}=\mathcal R_t\cup\{k<t:\ \Delta_t^{(k)}>\tau\}.

Replay rule (CoIN, seed 42)Experts replayedSciImNetVWGrndTxtVQAGQAVQAv2OCRAvg.
All earlier experts, round-robin (Algorithm 1)792.091.968.880.772.466.473.679.878.2
Measured forgetting (new)189.492.268.681.372.566.073.580.077.9

Replaying all experts in round-robin, as Algorithm 1 is written, reaches 78.2. The measured rule replays only ImageNet, the one task whose drift rises above τ, and reaches 77.9 without a hand choice: it keeps ImageNet at 92.2 while replaying one expert instead of seven. The revision adds the rule and this table to Algorithm 1 and Sec. 3.3.

W2: ReCaP’s benefits are grounded in being routerless and ease of deployment, since there is only one resulting checkpoint. There is no comparison with other methods or counterparts for deployment in terms of compute and memory. To be more specific, if one were to ask why this framework is beneficial for deployment, regardless of the number of checkpoints or experts, there is no clear answer.

A2: ReCaP serves at base-model cost, which expert pools cannot match. Measured on one A100, without any optimization:

MethodServed parametersPeak GPU memoryLatency, batch 1Throughput, batch 8
ReCaP3.75B7.1 GiB2.3 s2.84 req/s
LoRAMoE4.55B10.1 GiB5.0 s0.33 req/s
HiDe4.55B10.1 GiB4.5 s0.31 req/s
SOCiaL4.33B9.2 GiB4.6 s0.52 req/s

Expert pools keep every expert and router in memory; ReCaP serves the base model alone, so its cost does not grow with the number of tasks. Single-checkpoint methods share this cost, and ReCaP is the most accurate of them on every vision-language stream.

AuthorsOpenReview ↗

OpenReview title: Response to Reviewer Wntr (2/2): Task Order and Joint Training

Q1: Can you please explain what P_k is in section 3.2 ?

A3: P_k is the distribution of real task-k input-embedding blocks, the inputs that the task-k flow memory learns to reproduce. The revision defines it where it first appears in Sec. 3.2.

Q2: In the self-distillation comparison, can both methods be run under both task orders? Without order matching the current table supports no conclusion about self-distillation versus frozen-expert distillation.

A4: Yes; ReCaP stays ahead under both orders. We ran the two missing cells with Table 3's settings and evaluators:

Skill avg. / retention avg.science → tool → medicaltool → science → medical
ReCaP59.8 / 66.463.6 / 66.1 (new)
SDFT53.9 / 62.9 (new)52.0 / 62.0

ReCaP leads SDFT under both orders, on skills and on retention.

Q3: How was the joint multi-task reference model trained? (please refer to table 3, LLM 3-stage stream comparisons)

A5: Table 3 had no joint reference ("Base (no CL)" is the untrained Qwen2.5-7B-Instruct), so we trained one: one LoRA (rank 64, alpha 128) on the union of the science, tool and medical training splits, for 3,000 steps (the three experts' combined budget) at lr 1e-4, evaluated with Table 3's metrics.

ModelScienceToolMedicalSkill avg.Retention avg.
Joint reference (new)77.273.448.366.363.6
ReCaP (Table 3)76.061.841.659.866.4

The joint model learns the skills better, while ReCaP retains more general ability. The revision adds this row and its recipe to Table 3 and App. A.

Q4: Also from table 3, if a jointly trained multi-task model performs better than ReCaP trained experts, can you make a case for your method where a jointly trained model would be a more reasonable alternative over ReCaP?

A6: When all task data is available at once and can be kept, joint training is the better choice; ReCaP does not aim to replace it. ReCaP targets tasks that arrive in sequence, where earlier raw data cannot be kept:

CoINUCITUCIT-O Protocol 1Text-only (skill / retention)
Joint (all data at once)79.881.475.766.3 / 63.6
ReCaP (sequential, no stored data)78.081.075.559.8 / 66.4

Under that constraint ReCaP comes close to joint training on the vision-language streams and retains more general ability on the text-only stream. The two are complementary: ReCaP suits small, frequent model updates that add one task, and joint training suits large version updates that retrain on all data. The training cost of an update shows why:

CoIN training costJoint retrainingReCaP
Adding the 8th task64.4M tokens, 50.5 GPU-h9.2M tokens, 7.0 GPU-h
Whole 8-task stream261M tokens, 215.5 GPU-h76M tokens, 61.3 GPU-h
Earlier raw data neededYesNo

Joint retraining revisits all earlier data at every update, so its cost grows with the number of tasks; a ReCaP update trains one new expert and one consolidation stage. Joint cost is counted as one epoch over all data seen so far, at the per-sample cost of expert training.

We hope the all-experts ablation and forgetting-based selection, the measured deployment cost, the order-matched comparison with SDFT and the joint reference address the reviewer's concerns, and would appreciate the reviewer considering an increased score.

Reviewer_bfXZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReCaP distills frozen task-specific LoRA experts into one base-sized checkpoint, using flow matching to generate past-task input embeddings. Vision-language and text-only evaluations examine retention, with instruction-following degradation remaining in the text stream.

Critical strengths
  1. Tables 1–2 report 78.0 on CoIN and 75.5 on UCIT-O Protocol 1, versus SOCiaL's 77.4 and 70.7, while deploying one checkpoint without expert routing, directly supporting the practical deployment objective.
  2. Replay preserves ImageNet accuracy (35.4 to 92.5; Section 4.5 and Figure 3), while Table 4's held-out average changes from 58.9 to 58.7. The paper distinguishes retention from general-capability gains and acknowledges weaker text-stream results than EWC.
Critical weaknesses
  1. Table 4's “+distil” also includes label cross-entropy from Eq. (4), so gains over weight merging do not isolate expert distillation. Table 5 compares fidelity gaps and coverage, without establishing the downstream retention advantage of flow matching over simpler generators.
  2. Appendices A, G, and H are absent from the materials available for this review, leaving embedding interfaces, ρ/β, training budgets, and archive costs unverifiable and limiting assessment of reproducibility and training overhead.
Questions for authors
  1. Could you provide Appendices A, G, and H, or specify embedding-block construction and injection location, ρ/β, per-stage training budgets, and expert/generator archive sizes?
  2. Are there existing controls removing only the current-expert KL while retaining label cross-entropy, with student capacity, data, and update budgets fixed?
  3. Are downstream scores already available for GMM, Gaussian, and flow-matching replay under identical settings to clarify how their proxy differences translate into retention?
1 comment
AuthorsOpenReview ↗

Response to Reviewer bfXZ: KL Control, Downstream Generators and Implementation Details

We thank the reviewer for recognizing ReCaP's main strengths: with one checkpoint and no expert routing, it outperforms the expert-pool method SOCiaL on CoIN (78.0 vs 77.4) and UCIT-O Protocol 1 (75.5 vs 70.7), directly supporting the deployment objective; replay restores ImageNet from 35.4 to 92.5; and the paper separates prior-task retention from general capability and reports the text-only results candidly.

Q1/W2: Appendices A, G and H were not available, leaving the embedding-block construction, ρ\rho/β\beta, per-stage budgets and archive sizes unverifiable.

A1: They are in the separate supplementary PDF; the revision places all appendices after the references in one PDF. The requested details:

SettingVision-language streamsText-only stream
Current-task update probability ρ0.650.5
Label cross-entropy weight β1.01.0
Consolidation per stage500 steps, lr 1e-51,000 steps, lr 1e-5
Embedding blockInput to the first language-decoder layer: projected image features + question tokens (fixed image size and question length per task); no response tokensChat-formatted prompt, padded or truncated to 128 tokens; no response tokens
Injection during replayThe generated block is the input embedding of both expert and student; the frozen expert generates the continuation and the student matches its token distributionsSame
Archive per task (training only)223 MB expert + about 200 MB flow, under 6% of the 7.5 GB base646 MB expert + 206 MB flow (15.2 GB base)

Each consolidation step trains on the current task with probability ρ and otherwise replays an earlier task; β weights the label cross-entropy. Appendix A's 500 steps for the text-only stream is a typo for 1,000.

W1/Q2: Table 4's "+distil" also includes label cross-entropy from Eq. (4), so gains over weight merging do not isolate expert distillation. Are there existing controls removing only the current-expert KL while retaining label cross-entropy, with student capacity, data, and update budgets fixed?

A2: "+distil" combines both terms; we ran the requested control and will relabel the row. CoIN without replay; student capacity, data and the 500-step budget are fixed, and only the current-expert KL weight changes:

Objective (CoIN, no replay, seed 42)CoIN avg.OCR-VQAGrounding
Current-expert KL + label cross-entropy ("+distil")70.780.080.8
Label cross-entropy only (new)68.770.875.7

Removing only the current-expert KL lowers CoIN from 70.7 to 68.7, mostly on OCR-VQA and Grounding, so expert distillation contributes beyond label cross-entropy. Table 4 reports 70.9 for "+distil" as a 3-seed mean.

W1/Q3: Table 5 compares fidelity gaps and coverage, without establishing the downstream retention advantage of flow matching over simpler generators. Are downstream scores already available for GMM, Gaussian, and flow-matching replay under identical settings to clarify how their proxy differences translate into retention?

A3: These results were not available at submission; we ran them during the discussion period. Only the replay generator changes; experts, embedding blocks, replay share, student and steps are fixed:

Replay generator (seed 42)Size per taskCoIN avg.UCIT-O Protocol 1 avg.
Diagonal Gaussian5 MB77.474.9
GMM (5 components)316 MB78.175.2
Flow matching (ReCaP)About 200 MB78.175.6

Flow matching ties the larger GMM on CoIN and is best on UCIT-O. The revision adds this table to Sec. 4.5 and describes Table 5 as a fixed-student diagnostic.

We hope the KL control, the downstream generator comparison on two streams and the restored details address the reviewer's concerns, and would appreciate the reviewer considering an increased score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.