- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Action-Informed Merging (AIM), a training-free framework that integrates the domain-specific capabilities of multiple agent specialist models. It is initialized from a shared Qwen3.5-4B model into a single checkpoint without requiring inference-time routing.
It is mainly composed of the following two components: 1. Action-Informed Consensus Construction: It estimates parameter sensitivity (or importance) using the squared gradients of loss on action token spans (L$^act$) and loss on full responses (L$^{full}{_p}$ based on cross-expert support and sign agreement. 2. Sensitivity-Guided Update Selection: Via a diagonal quadratic surrogate objective function, it balances updates driven by action sensitivity toward specialist models with penalties driven by full-response sensitivity against an anchor checkpoint. Candidate parameter blocks are selected under explicit loss budgets (ϵ) for each experts.
This method is evaluated in two domains, tool-using and GUI, and both domains get improved.
Critical strengths
-
A sound framework based on action vs. response sensitivity: By decoupling action token loss gradients from full-response loss gradients, it provides an accurate method to maintain execution capabilities while suppressing divergence in general generation capabilities. Through the formulation of a quadratic surrogate objective function, the trade-off between maintaining anchor model performance and acquiring new capabilities from specialist models is explicitly modeled.
-
Broad interactive evaluation and high transparency: Verified across multifaceted interactive environments including function calling, terminal execution, web navigation, mobile operation, desktop OS tasks, and GUI grounding. Clear information disclosures are provided regarding evaluation protocols, prompt templates, calibration data sources, and the use of development sets.
Critical weaknesses
-
Generalisation concerns.
-
The largest reported average improvement is mainly driven by MobileWorld: the final balanced model scores below weighted averaging on web, desktop, and grounding tasks (as in Table 2). This trade-off matters when the goal is to combine several capabilities.
-
Lack of OOD evaluation on GUI benchmarks, which further addresses the concern. In Appendix A.4, authors explicitly stated that the 4-domain computer-use benchmarks were used as development data for model/setting selection. Because there are no results on an independent test set untouched by setting selection, it is unclear to what degree the reported 14.1% relative gain is due to generalizable consensus merging.
-
-
It is difficult to identify which parts of the method produce the improvements or to reproduce the final model.
- The final computer-use model combines the proposed merging approach with several additional mixing methods and refinement steps, while the tool-use model follows a different construction. The reported ablations mainly examine later refinements rather than separating the effects of the main proposed components (Table 4; Appendix D). A comparison with a simpler merging approach using the same specialists and comparable tuning would help establish the value of the action-focused design. A clearer end-to-end recipe, including the exact specialist checkpoints and configuration choices, is also needed for independent reproduction.
Questions for authors
-
Do evaluation results exist on unseen GUI/computer-use tasks that were completely unused during setting selection, anchor choice, and hyperparameter tuning?
-
What numerical value was set for κ in Equation (7) during the experiments? Also, what specific algorithm was used to solve the block selection optimization problem in Equation (9)?
-
In the overall consensus and selection pipeline, if the action mask loss L$^{act}$ is replaced with the regular response loss L$^{full}$ while holding other hyperparameters fixed, how does the performance of AIM change?