- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces Behavior-Aware Continual Agent Merging (BACAM), a continual agent merging method for interactive tasks that learns parameter-wise merging gates through expert-guided behavioral supervision on candidate-generated trajectories. The method consists of two main components: (1) Task-Level Stability–Plasticity Control, which includes New-task Plasticity Control, using the new expert to guide the acquisition of new capabilities, and Old-task Stability Control, using the previous merged model to preserve the existing capabilities; and (2) Tensor-Level Conflict-Aware Plasticity Budgeting, which restricts parameter updates in regions where new-task adaptation conflicts with the preservation of existing capabilities. Experiments are conducted across four interactive domains using specialized agents based on Qwen2.5-7B-Instruct. BACAM achieves an average success rate of 62.82%, outperforming the strongest evaluated merging baseline by 21.69%. Additional experiments investigate the effects of different merging orders and the contributions of the proposed components.
Critical strengths
-
The paper addresses an important problem: continual agent merging for multi-turn interactive tasks.
-
The experimental results are promising, showing substantial improvements over existing merging baselines. The merged model achieves competitive performance compared with the original experts, demonstrating its ability to preserve previously acquired capabilities.
Critical weaknesses
-
Lack of comparison with training methods like continual learning and on-policy distillation. BACAM is similar to on-policy distillation (OPD), but optimizes merging gates instead of directly updating model parameters. Most baselines are data-free or use limited training data. Without comparisons to direct continual fine-tuning or OPD, it is unclear whether the improvements come from on-policy supervision or the merging mechanism itself.
-
Insufficient evaluation of computational efficiency and scalability. Although BACAM introduces no additional test-time, it trains parameter-wise gates and requires repeated environment rollouts, teacher inference, and gradient updates. The paper lacks training cost comparisons, making it unclear whether BACAM is more efficient than direct fine-tuning or distillation. Its scalability to longer merging sequences also remains unclear.
Questions for authors
-
Why is parameter-wise model merging preferable to direct continual fine-tuning or on-policy distillation in the proposed setting, given that BACAM also requires gradient-based training and expert supervision?
-
What are the computational costs of BACAM compared with existing merging baselines and direct training methods, in terms of training time or GPU resources?
-
How well does BACAM scale to longer merging sequences with more diverse tasks? Could the authors consider additional interactive tasks, such as InterCode and τ-bench, to evaluate its scalability and robustness?
3 comments
W2, Q2 — Computational costs and scalability
Thank you for raising concerns about computational costs and scalability. Table 2 reports data use, training costs, and performance, with costs measured in GPU-hours. For 7B models with parameter-wise updates, we use four RTX 5090 GPUs with 32 GB each and FSDP ZeRO-3 sharding. This uses the same hardware resources as direct training. Most runtime is spent on multi-turn student inference and parameter updates. BACAM achieves better performance while maintaining comparable costs.
Table 3 further compares few-shot SFT and OPD without data replay against BACAM. With a small increase in inference cost, BACAM monitors old-task capabilities and balances performance on new and old tasks.
Table 2. Resource requirements and computational costs for merging four experts
| Method | Source model size | Trainable parameters | Data per task | Rollout cost | Training cost | Avg |
|---|---|---|---|---|---|---|
| WUDI-Merging | - | - | 0.48 | 41.13 | ||
| Expert Merging | 784 | 20 episodes | 0.21 | 3.60 | 17.22 | |
| One-shot | ||||||
| Continual Expert Merging | 1 episode | 0.35 | 15.29 | 41.72 | ||
| SFT | 1 episode | 0.38 | 12.55 | 37.96 | ||
| OPD | 1 episode | 0.98 | 10.11 | 43.36 | ||
| BACAM (ours) | 1 episode | 1.08 | 10.17 | 43.97 | ||
| Few-shot | ||||||
| Continual Expert Merging | 6 episodes | 0.27 | 14.71 | 54.44 | ||
| SFT | 6 episodes | 0.28 | 12.41 | 55.16 | ||
| OPD | 6 episodes | 0.85 | 9.71 | 57.89 | ||
| BACAM (ours) | 6 episodes | 0.97 | 9.65 | 58.70 |
Note: denotes the parameter count of one model. Costs are in GPU-hours. Few-shot data are more diverse and yield better model performance. This reduces unproductive loops and excessively long outputs, leading to lower runtime than one-shot.
Table 3. Costs and performance of data replay in few-shot continual training
| Method | Source model size | Trainable parameters | Data per task | Data replay | Rollout cost | Training cost | Avg |
|---|---|---|---|---|---|---|---|
| SFT | 6 episodes | off | 0.12 | 8.26 | 54.41 | ||
| OPD | 6 episodes | off | 0.46 | 8.92 | 56.16 | ||
| BACAM (ours) | 6 episodes | on | 0.97 | 9.65 | 58.70 |
W2, Q3 — Scalability to longer merging sequences
Thank you for raising the issue of generalization. We evaluated a longer merging sequence, as shown in Table 1. Given the time available, we added the Airline task from τ²-bench. The five-stage order is WebShop, Tool, Search, ALFWorld, and τ²-bench. The results show limited performance loss on old tasks and effective acquisition of the new task.
Table 1. Performance over a longer merging sequence
| Method | WebShop | Tool | Search | ALFWorld IID | ALFWorld OOD | τ²-bench |
|---|---|---|---|---|---|---|
| BACAM few-shot T4 (ours) | 76.80 | 28.00 | 23.15 | 86.43 | 79.10 | - |
| BACAM few-shot T5 (ours) | 75.40 | 25.00 | 21.72 | 79.29 | 74.63 | 40.91 |
| Forgetting | 1.40 | 3.00 | 1.44 | 7.14 | 4.48 | - |
Note: The extended experiment uses the following benchmark, evaluation framework, and expert.
-
Benchmark. We use the Airline task from τ²-bench. τ²-bench paper
-
Evaluation framework. We use the τ²-bench implementation provided by Simia. Both the user simulator and the natural-language judge use the qwen3.8-max API. Simia paper
-
Expert checkpoint. We use Simia-Tau-SFT-RL-Qwen2.5-7B. Its Airline success rate under these settings is 47.73%. Hugging Face checkpoint
W1, Q1 — Fairness of baseline comparisons
Thank you for raising concerns about the fairness of baseline comparisons. We added continual supervised fine-tuning (SFT) and on-policy distillation (OPD) baselines with matched data amounts and numbers of trainable parameters. We also compare against WUDI-Merging and Continual Expert Merging, which adapts Expert Merging to the continual setting. Table 1 reports the per-task results and shows the contribution of the merging mechanism.
We think performance depends in part on the amount and diversity of supervision. All data-based baselines perform better with few-shot supervision than with one-shot supervision.
For SFT, teacher-generated trajectories offer limited diversity. BACAM uses student-generated trajectories and outperforms SFT in complex multi-turn agent tasks, especially on OOD tasks.
For OPD, BACAM also achieves better performance with limited supervision. Excluding the benefits of improved supervision, we attribute these gains over OPD to expert model interpolation, which provides prior information, and other modules such as task-level behavioral control and tensor-level conflict control.
Table 1. Per-task performance