acceptodds
Under review as a conference paper at ICLR 2027

BACAM: Behavior-Aware Continual Agent Merging For Multi-turn Interaction

Abstract

Model merging offers a way to integrate the capabilities of specialized experts, but existing agent merging methods typically require all of them to be available at once. We study continual agent merging, which integrates incoming experts sequentially without retaining previously merged experts. Yet merging in parameter space or feature subspaces does not ensure that the merged model acquires an incoming expert's behavior on interaction trajectories. Moreover, updates toward a new expert can disrupt the merged model's previously integrated interactive behavior. Therefore, we propose Behavior-Aware Continual Agent Merging (BACAM), which learns parameter-wise merging gates from candidate-generated trajectories using expert-guided behavioral supervision. Task-level stability–plasticity control and tensor-level conflict-aware update budgets limit interference with existing capabilities while allowing new ones to be acquired. The learned gates are folded into the model weights without additional inference-time parameters. Across four interactive tasks—web shopping, tool use, information retrieval, and embodied interaction—BACAM achieves an average success rate of 62.82%, exceeding the strongest evaluated merging baseline by 21.69 percentage points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
model merging, agent, multi-turn interaction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vJgcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Behavior-Aware Continual Agent Merging (BACAM), a continual agent merging method for interactive tasks that learns parameter-wise merging gates through expert-guided behavioral supervision on candidate-generated trajectories. The method consists of two main components: (1) Task-Level Stability–Plasticity Control, which includes New-task Plasticity Control, using the new expert to guide the acquisition of new capabilities, and Old-task Stability Control, using the previous merged model to preserve the existing capabilities; and (2) Tensor-Level Conflict-Aware Plasticity Budgeting, which restricts parameter updates in regions where new-task adaptation conflicts with the preservation of existing capabilities. Experiments are conducted across four interactive domains using specialized agents based on Qwen2.5-7B-Instruct. BACAM achieves an average success rate of 62.82%, outperforming the strongest evaluated merging baseline by 21.69%. Additional experiments investigate the effects of different merging orders and the contributions of the proposed components.

Critical strengths
  1. The paper addresses an important problem: continual agent merging for multi-turn interactive tasks.

  2. The experimental results are promising, showing substantial improvements over existing merging baselines. The merged model achieves competitive performance compared with the original experts, demonstrating its ability to preserve previously acquired capabilities.

Critical weaknesses
  1. Lack of comparison with training methods like continual learning and on-policy distillation. BACAM is similar to on-policy distillation (OPD), but optimizes merging gates instead of directly updating model parameters. Most baselines are data-free or use limited training data. Without comparisons to direct continual fine-tuning or OPD, it is unclear whether the improvements come from on-policy supervision or the merging mechanism itself.

  2. Insufficient evaluation of computational efficiency and scalability. Although BACAM introduces no additional test-time, it trains parameter-wise gates and requires repeated environment rollouts, teacher inference, and gradient updates. The paper lacks training cost comparisons, making it unclear whether BACAM is more efficient than direct fine-tuning or distillation. Its scalability to longer merging sequences also remains unclear.

Questions for authors
  1. Why is parameter-wise model merging preferable to direct continual fine-tuning or on-policy distillation in the proposed setting, given that BACAM also requires gradient-based training and expert supervision?

  2. What are the computational costs of BACAM compared with existing merging baselines and direct training methods, in terms of training time or GPU resources?

  3. How well does BACAM scale to longer merging sequences with more diverse tasks? Could the authors consider additional interactive tasks, such as InterCode and τ-bench, to evaluate its scalability and robustness?

3 comments

W1, Q1 — Fairness of baseline comparisons

Thank you for raising concerns about the fairness of baseline comparisons. We added continual supervised fine-tuning (SFT) and on-policy distillation (OPD) baselines with matched data amounts and numbers of trainable parameters. We also compare against WUDI-Merging and Continual Expert Merging, which adapts Expert Merging to the continual setting. Table 1 reports the per-task results and shows the contribution of the merging mechanism.

We think performance depends in part on the amount and diversity of supervision. All data-based baselines perform better with few-shot supervision than with one-shot supervision.

For SFT, teacher-generated trajectories offer limited diversity. BACAM uses student-generated trajectories and outperforms SFT in complex multi-turn agent tasks, especially on OOD tasks.

For OPD, BACAM also achieves better performance with limited supervision. Excluding the benefits of improved supervision, we attribute these gains over OPD to expert model interpolation, which provides prior information, and other modules such as task-level behavioral control and tensor-level conflict control.

Table 1. Per-task performance

MethodData per taskData replayWebShopToolSearchALFWorld IIDALFWorld OODAvg
WUDI-Merging--38.200.0025.1472.1470.1541.13
Expert Merging20 episodes-5.601.0018.8428.5732.0917.22
One-shot
Continual Expert Merging1 episodeon77.4025.0020.1845.0041.0441.72
SFT1 episodeon75.2033.0018.3937.8625.3737.96
OPD1 episodeon77.4017.0021.8654.2946.2743.36
BACAM (ours)1 episodeon77.8014.0020.7252.8654.4843.97
Few-shot
Continual Expert Merging6 episodeson78.4025.0023.5572.8672.3954.44
SFT6 episodesoff77.2030.0022.6173.5768.6654.41
SFT6 episodeson78.8033.0023.2072.1468.6655.16
OPD6 episodesoff79.2026.0023.9078.5773.1356.16
OPD6 episodeson77.2027.0023.9581.4379.8557.89
BACAM (ours)6 episodeson76.8028.0023.1586.4379.1058.70

W2, Q2 — Computational costs and scalability

Thank you for raising concerns about computational costs and scalability. Table 2 reports data use, training costs, and performance, with costs measured in GPU-hours. For 7B models with parameter-wise updates, we use four RTX 5090 GPUs with 32 GB each and FSDP ZeRO-3 sharding. This uses the same hardware resources as direct training. Most runtime is spent on multi-turn student inference and parameter updates. BACAM achieves better performance while maintaining comparable costs.

Table 3 further compares few-shot SFT and OPD without data replay against BACAM. With a small increase in inference cost, BACAM monitors old-task capabilities and balances performance on new and old tasks.

Table 2. Resource requirements and computational costs for merging four experts

MethodSource model sizeTrainable parametersData per taskRollout costTraining costAvg
WUDI-Merging5P5P≈P\approx P--0.4841.13
Expert Merging5P5P78420 episodes0.213.6017.22
One-shot
Continual Expert Merging2P2PPP1 episode0.3515.2941.72
SFT2P2PPP1 episode0.3812.5537.96
OPD2P2PPP1 episode0.9810.1143.36
BACAM (ours)2P2PPP1 episode1.0810.1743.97
Few-shot
Continual Expert Merging2P2PPP6 episodes0.2714.7154.44
SFT2P2PPP6 episodes0.2812.4155.16
OPD2P2PPP6 episodes0.859.7157.89
BACAM (ours)2P2PPP6 episodes0.979.6558.70

Note: PP denotes the parameter count of one model. Costs are in GPU-hours. Few-shot data are more diverse and yield better model performance. This reduces unproductive loops and excessively long outputs, leading to lower runtime than one-shot.

Table 3. Costs and performance of data replay in few-shot continual training

MethodSource model sizeTrainable parametersData per taskData replayRollout costTraining costAvg
SFT2P2PPP6 episodesoff0.128.2654.41
OPD2P2PPP6 episodesoff0.468.9256.16
BACAM (ours)2P2PPP6 episodeson0.979.6558.70

W2, Q3 — Scalability to longer merging sequences

Thank you for raising the issue of generalization. We evaluated a longer merging sequence, as shown in Table 1. Given the time available, we added the Airline task from τ²-bench. The five-stage order is WebShop, Tool, Search, ALFWorld, and τ²-bench. The results show limited performance loss on old tasks and effective acquisition of the new task.

Table 1. Performance over a longer merging sequence

MethodWebShopToolSearchALFWorld IIDALFWorld OODτ²-bench
BACAM few-shot T4 (ours)76.8028.0023.1586.4379.10-
BACAM few-shot T5 (ours)75.4025.0021.7279.2974.6340.91
Forgetting1.403.001.447.144.48-

Note: The extended experiment uses the following benchmark, evaluation framework, and expert.

  1. Benchmark. We use the Airline task from τ²-bench. τ²-bench paper

  2. Evaluation framework. We use the τ²-bench implementation provided by Simia. Both the user simulator and the natural-language judge use the qwen3.8-max API. Simia paper

  3. Expert checkpoint. We use Simia-Tau-SFT-RL-Qwen2.5-7B. Its Airline success rate under these settings is 47.73%. Hugging Face checkpoint

Reviewer_ygNHOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes BACAM for continual agent merging, where specialized LLM agent experts are merged one at a time into a single model and earlier experts are discarded. BACAM learns a parameter-wise gate between the current merged model and the incoming expert, trained on trajectories generated by the candidate model: forward KL to the incoming expert on the new task, reverse KL to the previous merged model on earlier tasks (enforced as per-task constraints), and a tensor-level budget that limits updates where gradients conflict. Four Qwen2.5-7B-Instruct experts (WebShop, tool use, search, ALFWorld) are merged and evaluated on WebShop, BFCL v3 multi-turn, MuSiQue, and ALFWorld. In the default order, BACAM reaches 62.82% average success rate versus 41.13% for the strongest baseline.

Critical strengths
  1. The paper formulates a setting that prior work has not addressed: merging interactive agent experts one at a time, with earlier experts discarded after each merge. Existing agent-merging methods assume all experts are available at once, and existing continual-merging methods are evaluated on non-interactive tasks.
  2. The evaluation is broad. It covers many general, agent-specific, and continual merging baselines, four different arrival orders with results tracked after every merge, ablations of each component, and diagnostics of how the gates and budgets behave across layers and tensor types.
  3. The gains are consistent rather than confined to the final model. The method has the highest average success rate after two, three, and four experts in the default order, and on tool use it reaches 32–38% where every other merging method stays at 0–2%.
  4. The learned gates are folded back into the weights, so the result is a single standard checkpoint with no added parameters or inference-time cost, unlike the adapter- or expert-based continual merging methods it compares against. The same hyperparameters are used across all four arrival orders.
Critical weaknesses
  1. The comparison with baselines is not budget-matched. Appendix B.2 states that data collection and update schedules are method-specific, and the proposed method is the only one that interacts with the environments during merging.
  2. Performance depends on the order in which experts arrive. Final average success rate ranges from 56.02% to 62.82% across the four orders, and WebShop alone varies by 20.40 points. The authors list robustness to order as future work.
  3. Not every capability is fully acquired. ALFWorld, integrated last in the default order, remains 10.00 and 4.47 points below its expert on the IID and OOD splits, and Search is slightly below the best baseline.
  4. The tensor-level budget does not help uniformly. Removing it slightly improves WebShop while lowering the other tasks.
Questions for authors
  1. Do the authors have an explanation for why WebShop and ALFWorld are sensitive to arrival order while Tool and Search are not?
  2. Since the method requires environment access and gradient-based optimization at merge time, how does it compare to multi-task fine-tuning or multi-teacher on-policy distillation using the same rollouts and teachers?
6 comments

Thank you for your constructive comments. We address your concerns below.

W1 and Q2 — Fairness of baseline comparisons and additional experiments

Thank you for raising the issue of fairness and suggesting additional experiments. Expert Merging also uses calibration data for its updates and interacts with the environment in our comparison. Our method uses a different data collection strategy to guide parameter interpolation. We have clarified this misinterpretation in the paper. Table 1 adds budget-matched comparisons with gradient-based multi-task fine-tuning (SFT) and multi-teacher on-policy distillation (OPD). Table 2 reports per-task scores.

The results show that performance depends on the amount and diversity of data under the same budget. The merging mechanism provides further gains. Teacher-generated trajectories offer limited diversity. BACAM uses student-generated trajectories and outperforms SFT in complex multi-turn agent tasks, especially on OOD tasks. With limited supervision, BACAM also outperforms OPD. We attribute this to the prior provided by interpolation between expert models, task-level behavioral control, and tensor-level conflict control.

Table 1. Resource requirements and computational costs for merging four experts

MethodSource model sizeTrainable parametersData per taskRollout costTraining costAvg
WUDI-Merging5P5P≈P\approx P--0.4841.13
One-shot
Continual Expert Merging2P2PPP1 episode0.3515.2941.72
SFT2P2PPP1 episode0.3812.5537.96
OPD2P2PPP1 episode0.9810.1143.36
BACAM (ours)2P2PPP1 episode1.0810.1743.97
Few-shot
Continual Expert Merging2P2PPP6 episodes0.2714.7154.44
SFT2P2PPP6 episodes0.2812.4155.16
OPD2P2PPP6 episodes0.859.7157.89
BACAM (ours)2P2PPP6 episodes0.979.6558.70

Note: PP denotes the parameter count of one model. Costs are in GPU-hours. Few-shot data are more diverse and yield better model performance. This reduces unproductive loops and excessively long outputs, leading to lower runtime than one-shot.

Table 2. Per-task performance

MethodWebShopToolSearchALFWorld IIDALFWorld OODAvg
WUDI-Merging38.200.0025.1472.1470.1541.13
One-shot
Continual Expert Merging77.4025.0020.1845.0041.0441.72
SFT75.2033.0018.3937.8625.3737.96
OPD77.4017.0021.8654.2946.2743.36
BACAM (ours)77.8014.0020.7252.8654.4843.97
Few-shot
Continual Expert Merging78.4025.0023.5572.8672.3954.44
SFT78.8033.0023.2072.1468.6655.16
OPD77.2027.0023.9581.4379.8557.89
BACAM (ours)76.8028.0023.1586.4379.1058.70

W2 and Q1 — Sensitivity to merging order

Thank you for raising the issue of merging order. We attribute order sensitivity to the differences in new-task capability acquisition across tasks. Our method remains robust across the merging orders. The order determines the initial expert and which capabilities must be acquired later. For example, in WebShop -> Tool, WebShop is the initial expert and Tool is the new task. In the reverse order, Tool is the initial expert and requires acquiring WebShop. Table 1 shows lower acquisition relative to the expert for WebShop (83.93%) and ALFWorld (90.34% IID and 92.63% OOD) than for Search (100.90%) and Tool (106.67%), which is caused by differences in trajectory coverage and how clearly teacher supervision distinguishes between decisions across tasks. Within each task, performance at the integration stage shows limited variation across merging orders, with a maximum standard deviation of about 1.96 percentage points. Table 2 shows limited forgetting across the merging orders. Together, these results suggest that order sensitivity mainly concerns the differences in new-task capability acquisition across tasks, while capability acquisition within each task and retention of existing capabilities remain robust across merging orders.

Table 1. New-task capability acquisition across merging orders

Merging orderWebShopToolSearchALFWorld IIDALFWorld OOD
WTSAInitial expert32.00 (T2)24.10 (T3)88.57 (T4)86.57 (T4)
ASTW63.00 (T4)31.00 (T3)24.14 (T2)Initial expertInitial expert
TWAS67.80 (T2)Initial expert25.43 (T4)88.57 (T3)82.84 (T3)
SAWT65.60 (T3)33.00 (T4)Initial expert90.00 (T2)83.58 (T2)
Mean ± SD65.47 ± 1.9632.00 ± 0.8224.56 ± 0.6289.05 ± 0.6784.33 ± 1.61
Capability acquisition rate83.93%106.67%100.90%90.34%92.63%

Note: Tn denotes the stage at which a task is introduced. Capability acquisition measures performance on a newly integrated task relative to its expert. The expert reference scores are 78.00, 30.00, 24.34, 98.57, and 91.04, respectively.

Table 2. Forgetting

Merging orderInitial taskExpert scoreFinal scoreDrop (percentage points)Retention
WTSAWebShop78.0077.200.8098.97%
ASTWALFWorld IID98.5796.432.1497.83%
ASTWALFWorld OOD91.0491.040.00100.00%
TWASTool30.0038.00−8.00126.67%
SAWTSearch24.3425.43−1.09104.49%

Note: Each order reports forgetting only for its initial task.

W3 — Merged model performance

Thank you for pointing out this performance gap. In our main experiment, BACAM scores 23.75 on Search, compared with 25.14 for WUDI-Merging, the best-performing baseline, a gap of 1.39 percentage points. BACAM retains 97.58% of the Search expert score. After integrating the other tasks, BACAM achieves an average of 62.82 across the five metrics, compared with 41.13 for WUDI-Merging. These results show stronger overall capabilities, while Search retention remains an area for improvement.

Table 1. Performance of merged models, baselines, and experts

MethodWebShopToolSearchALFWorld IIDALFWorld OODAvg
Search expert0.8011.0024.349.2910.4511.18
ALFWorld expert1.8016.0014.9298.5791.0444.47
WUDI-Merging38.200.0025.1472.1470.1541.13
BACAM (ours)77.2038.0023.7588.5786.5762.82
Reviewer_ypeeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Behavior-Aware Continual Agent Merging (BACAM), a framework for sequentially integrating specialized LLM agents without retaining previously merged experts. The authors point out that existing model merging methods mainly focus on parameter-level or feature-level integration, which may not preserve an agent’s behavior during multi-turn interactions. To address this issue, BACAM learns parameter-wise merging gates using behavioral supervision collected from candidate-generated interaction trajectories. The framework combines task-level stability–plasticity control to balance new capability acquisition with the preservation of earlier behaviors, and tensor-level conflict-aware plasticity budgeting to reduce interference between tasks. The main contribution is the idea of making continual agent merging explicitly behavior-aware rather than relying only on parameter similarity or feature-space constraints. For validation, the authors evaluate BACAM using four specialized agents built on Qwen2.5-7B-Instruct, covering web shopping, tool use, information retrieval, and embodied interaction. The experiments compare BACAM against several conventional, agent-specific, and continual model merging baselines. BACAM achieves an average success rate of 62.82%, outperforming the strongest evaluated baseline by 21.69 percentage points. The authors also examine different expert arrival orders, capability retention across merging stages, and the contributions of individual components through ablation studies. These results suggest that behavior-guided merging can improve capability integration and reduce forgetting in interactive agents, although the observed sensitivity to merging order and the evaluation on a single backbone leave some questions about its broader generalizability.

Critical strengths

First, I think one important strength of the paper is its focus on continual agent merging in multi-turn interactive environments. Unlike conventional model merging methods that mainly consider parameter integration or static prediction tasks, the authors explicitly account for how an agent’s actions influence subsequent observations and decisions. I appreciate the use of candidate-generated trajectories to guide behavioral alignment with incoming experts while preserving previously acquired capabilities. The combination of task-level stability–plasticity control and tensor-level conflict-aware updates provides a reasonable framework for reducing interference during sequential merging. This is meaningful because maintaining consistent behavior across multiple interaction steps is particularly important for practical LLM agents. Second, the experimental results are encouraging, particularly the improvements over a relatively broad set of model merging baselines. The reported average success rate of 62.82%, exceeding the strongest evaluated baseline by 21.69 percentage points, suggests that explicitly incorporating behavioral supervision can substantially improve merging performance. I also appreciate that the authors evaluate four different interactive tasks, examine multiple expert arrival orders, and provide ablation studies to investigate the contributions of their proposed components. These experiments provide useful evidence that the method can acquire new capabilities while retaining earlier ones, making the empirical performance a significant positive aspect of the paper.

Critical weaknesses

First, my main concern is the fairness of the experimental comparisons and whether the reported improvements can be attributed to the proposed merging mechanism itself. BACAM uses candidate-generated interaction trajectories, behavioral supervision from the incoming expert and current merged model, and iterative gate optimization. However, many of the strongest baselines operate directly on model parameters without access to comparable behavioral supervision or interaction data. This creates a substantial difference in the information and computational resources available to each method. Although the reported performance improvements are impressive, it is difficult to determine how much of the improvement comes from the proposed stability–plasticity and conflict-aware mechanisms rather than the additional behavioral supervision. I would like to see stronger comparisons with data-assisted merging or distillation approaches under matched supervision and computational budgets. Without these comparisons, I am not fully convinced that the experiments establish the methodological advantage of BACAM.

Second, I am concerned about the limited generalizability of the experimental findings and the practical scalability of the proposed framework. The evaluation uses only one model backbone, Qwen2.5-7B-Instruct, and four specialized experts, which makes it difficult to assess whether the approach would remain effective across different model architectures, larger models, or longer sequences of incoming experts. More importantly, BACAM requires access to previously encountered task instances and environments, repeated interaction rollouts, and behavioral evaluations during each merging stage. These requirements may become increasingly expensive as the number of integrated tasks grows, potentially limiting the practical benefits of continual merging. The paper also reports noticeable sensitivity to expert arrival order, suggesting that capability acquisition and retention are not consistently robust. I believe broader cross-model experiments, longer merging sequences, and a detailed analysis of training and interaction costs are necessary to establish the scalability and practical significance of the approach.

Questions for authors

I have several questions that could materially affect my assessment of the paper. First, could the authors provide a more comprehensive factorial ablation combining the task-level KL choices (forward, reverse, and asymmetric), CAPB on/off, and token-group weighting on/off? The current results show relatively modest improvements from individual components, particularly the 1.24-point gain from CAPB, and I would like to understand whether these mechanisms provide distinct benefits or whether similar performance could be achieved through better tuning of the stability budget ε. Second, I am concerned about the fairness of the baseline comparisons. Since BACAM benefits from candidate-generated multi-turn trajectories and expert-guided behavioral supervision, could the authors clarify whether its 21.69-point improvement over WUDI-Merging would remain significant if WUDI-Merging or Expert Merging were given comparable interaction data, teacher distributions, and computational budgets? This comparison would help establish whether the improvement comes primarily from the proposed merging mechanism or from access to richer supervision. Third, could the authors explain the substantial sensitivity to expert arrival order, particularly why integrating WebShop later results in a 10.20–15.00-point performance gap and why the final average success rate varies by 6.80 points across different orders? I would be interested in whether these differences arise from gate saturation, Lagrangian-multiplier dynamics, or the interaction between token-group weighting and relatively sparse action tokens. Fourth, could the authors confirm that all baselines were evaluated using identical held-out instances, generation settings, and evaluation procedures, including temperature and maximum output tokens? It would also be helpful to clarify whether all baseline results were reproduced using the same evaluation framework or taken from previous studies, especially given the differences in ALFWorld interaction rounds. Finally, Table 6 shows that BACAM achieves 32.00% Tool success after integrating only two experts, exceeding the standalone Tool expert’s 30.00%. Could the authors clarify whether this improvement reflects positive transfer from WebShop or experimental variability? I believe convincing answers to the questions about baseline fairness and merging-order sensitivity, in particular, could significantly improve my assessment. As separate suggestions for future work, evaluating an additional model backbone, extending the merging sequence beyond four experts, and reporting per-stage computational costs would further strengthen the paper’s generalizability and practical relevance.

6 comments

Thank you for your constructive comments. We address your concerns below.

W1 and Q2 — Fairness of baseline comparisons

Thank you for raising the issue of fair comparison. We compare resource use across different amounts of supervision and examine the contributions of the merging mechanism and supervision. Table 1 reports source model size, trainable parameters, interaction data, rollout costs, and training costs. Table 2 reports per-task results. We will add these comparisons to the revised version in blue.

WUDI-Merging minimizes interference using expert task vectors and optimizes merging weights matrix by matrix. Adding behavioral supervision would change the original method. Expert Merging and BACAM use the same calibration split. We also adapt Expert Merging to continual merging and match the number of trainable parameters. The tables call this variant Continual Expert Merging.

We added on-policy distillation (OPD) and supervised fine-tuning (SFT) baselines with data replay to examine the contributions of behavioral supervision and merging. The results show that performance depends on the amount and diversity of data under the same budget. The merging mechanism provides further gains. Teacher-generated trajectories offer limited diversity. BACAM uses student-generated trajectories and outperforms SFT in complex multi-turn agent tasks, especially on OOD tasks. With limited supervision, BACAM also outperforms OPD. We attribute this to the prior provided by interpolation between expert models, task-level behavioral control, and tensor-level conflict control.

Table 1. Resource requirements and computational costs for merging four experts

MethodSource model sizeTrainable parametersData per taskRollout costTraining costAvg
WUDI-Merging5P5P≈P\approx P--0.4841.13
One-shot
Continual Expert Merging2P2PPP1 episode0.3515.2941.72
SFT2P2PPP1 episode0.3812.5537.96
OPD2P2PPP1 episode0.9810.1143.36
BACAM (ours)2P2PPP1 episode1.0810.1743.97
Few-shot
Continual Expert Merging2P2PPP6 episodes0.2714.7154.44
SFT2P2PPP6 episodes0.2812.4155.16
OPD2P2PPP6 episodes0.859.7157.89
BACAM (ours)2P2PPP6 episodes0.979.6558.70

Note: PP denotes the parameter count of one model. Costs are in GPU-hours. Few-shot data are more diverse and yield better model performance. This reduces unproductive loops and excessively long outputs, leading to lower runtime than one-shot.

Table 2. Per-task performance

MethodWebShopToolSearchALFWorld IIDALFWorld OODAvg
WUDI-Merging38.200.0025.1472.1470.1541.13
One-shot
Continual Expert Merging77.4025.0020.1845.0041.0441.72
SFT75.2033.0018.3937.8625.3737.96
OPD77.4017.0021.8654.2946.2743.36
BACAM (ours)77.8014.0020.7252.8654.4843.97
Few-shot
Continual Expert Merging78.4025.0023.5572.8672.3954.44
SFT78.8033.0023.2072.1468.6655.16
OPD77.2027.0023.9581.4379.8557.89
BACAM (ours)76.8028.0023.1586.4379.1058.70

W1 and Q1 — Ablations of BACAM components

Thank you for asking about component effectiveness. We provide more detailed ablations of the amount of supervision and component ablations. The tables below report the results. We will update the original ablation table and mark the additions in blue.

Table 1 shows that data amount strongly affects merging performance. Broader coverage of behavioral supervision leads to better results. Table 2 presents more comprehensive component ablations, demonstrating the effectiveness of BACAM’s components. Table 3 further supports task-level stability–plasticity control and tensor-level conflict-aware plasticity budgeting. Adjusting the stability budget ε alone does not achieve the same performance without CAPB. Our setting is ε=0.005 and CAPB is on.

Table 1. Ablation of data amount

MethodData per taskWebShopToolSearchALFWorld IIDALFWorld OODAvg
BACAM one-shot (ours)1 episode77.8014.0020.7252.8654.4843.97
BACAM few-shot (ours)6 episodes76.8028.0023.1586.4379.1058.70
BACAM (ours)20 episodes77.2038.0023.7588.5786.5762.82

Table 2. Component ablations

MethodLossKLCAPBTGAvg
EqualSPCForwardReverseAsymmetricOnOffOnOff
BACAM (ours)✓×××✓✓×✓×61.42
BACAM (ours)×✓✓××✓×✓×60.89
BACAM (ours)×✓×✓×✓×✓×62.14
BACAM (ours)×✓××✓✓××✓61.40
BACAM (ours)×✓××✓×✓✓×61.58
BACAM (ours)×✓××✓✓×✓×62.82

Note: Equal denotes equal weighting of new- and old-task losses. SPC denotes Task-Level Stability–Plasticity Control. Asymmetric KL uses forward KL for new tasks and reverse KL for old tasks. CAPB denotes Tensor-Level Conflict-Aware Plasticity Budgeting. TG denotes Token-group weighting.

Table 3. Ablations of task-level stability control and tensor-level conflict control

CAPBε=0.001ε=0.005ε=0.01ε=0.05
on62.1162.8261.0058.76
off-61.58--

W2 and Q3 — Sensitivity to merging order

Thank you for raising the issue of merging order. We attribute order sensitivity to the differences in new-task capability acquisition across tasks, which is not caused by gate saturation, Lagrangian-multiplier dynamics, the interaction between token-group weighting and relatively sparse action tokens. Our method remains robust across the merging orders. The order determines the initial expert and which capabilities must be acquired later. For example, in WebShop -> Tool, WebShop is the initial expert and Tool is the new task. In the reverse order, Tool is the initial expert and requires acquiring WebShop. Table 1 shows lower acquisition relative to the expert for WebShop (83.93%) and ALFWorld (90.34% IID and 92.63% OOD) than for Search (100.90%) and Tool (106.67%), which is caused by differences in trajectory coverage and how clearly teacher supervision distinguishes between decisions across tasks. Within each task, performance at the integration stage shows limited variation across merging orders, with a maximum standard deviation of about 1.96 percentage points. Table 2 shows limited forgetting across the merging orders. Together, these results suggest that order sensitivity mainly concerns the differences in new-task capability acquisition across tasks, while capability acquisition within each task and retention of existing capabilities remain robust across merging orders.

Table 1. New-task capability acquisition across merging orders

Merging orderWebShopToolSearchALFWorld IIDALFWorld OOD
WTSAInitial expert32.00 (T2)24.10 (T3)88.57 (T4)86.57 (T4)
ASTW63.00 (T4)31.00 (T3)24.14 (T2)Initial expertInitial expert
TWAS67.80 (T2)Initial expert25.43 (T4)88.57 (T3)82.84 (T3)
SAWT65.60 (T3)33.00 (T4)Initial expert90.00 (T2)83.58 (T2)
Mean ± SD65.47 ± 1.9632.00 ± 0.8224.56 ± 0.6289.05 ± 0.6784.33 ± 1.61
Capability acquisition rate83.93%106.67%100.90%90.34%92.63%

Note: Tn denotes the stage at which a task is introduced. Capability acquisition measures performance on a newly integrated task relative to its expert. The expert reference scores are 78.00, 30.00, 24.34, 98.57, and 91.04, respectively.

Table 2. Forgetting

Merging orderInitial taskExpert scoreFinal scoreDrop (percentage points)Retention
WTSAWebShop78.0077.200.8098.97%
ASTWALFWorld IID98.5796.432.1497.83%
ASTWALFWorld OOD91.0491.040.00100.00%
TWASTool30.0038.00−8.00126.67%
SAWTSearch24.3425.43−1.09104.49%

Note: Each order reports forgetting only for its initial task.

Comments on the paper

3 comments

In addition, we are supplementing experiments on the four-task model merging based on the Llama architecture, covering WebShop, Tool, Search, and ALFWorld.

Update: We have included the experiments with the Llama architecture in blue in the revised paper.

We thank the reviewers for their insightful comments. We have carefully considered their comments and provided detailed responses. We look forward to their feedback.

We thank all reviewers for their constructive feedback. In response, we revised the manuscript and added the following analyses and experiments:

Baseline fairness and computational cost — Reviewers ypee, ygnh, and vJgc. We clarified the data and supervision used by each baseline and documented the shared evaluation protocol. In our responses, we added comparisons with continual SFT, on-policy distillation, and Expert Merging under matched data budgets, and reported rollout and training costs separately.

Behavioral constraints and conflict-aware budgets — Reviewers ypee and ygnh. We expanded the ablations to cover equal task weighting, KL direction, token-group weighting, and the conflict-aware plasticity budget (CAPB). We also varied the task-level stability budget with CAPB enabled and compared one-shot, few-shot, and full-data supervision.

Merging order and capability retention — Reviewers ypee and ygnh. We reported new-task capability acquisition and forgetting across four merging orders. Acquisition differed across tasks, while each task’s score at the stage when it was introduced varied less across orders.

Generalization across backbones and longer sequences — Reviewers ypee and vJgc. We added a Llama-3.1-8B-Instruct experiment alongside the Qwen results. BACAM achieved an average score of 49.52%, compared with 17.65% for the strongest evaluated merging baseline on Llama. We also extended the Qwen sequence to five experts by adding τ²-bench Airline and evaluated both Airline performance and retention of the four earlier tasks.

We appreciate the reviewers’ suggestions, which helped us clarify the evaluation and better characterize when BACAM acquires and retains agent capabilities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.