- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MATE extends MASS for dynamic model merging when the task identity is unknown. It uses a small proxy set to select routing layers according to a relative projection residual margin, averages their task-relevance scores, and reuses the aggregated scores to guide classification-head selection. The task-subspace representation, adaptive merging procedure, and two-pass inference structure follow MASS. The paper evaluates these changes across three CLIP ViT backbones and three task counts.
Critical strengths
The changes are simple and improve on MASS in all nine evaluated settings. Some improvements are meaningful: ViT-B/32 with 14 tasks increases from 82.90 to 85.09, and ViT-B/16 with 20 tasks increases from 81.11 to 82.95. The routing analysis and component ablation provide useful evidence about how the changes affect task identification and final classification. The supplementary material improves the transparency of the work. It includes the implementation, configurations, layer-wise proxy statistics, and per-task evaluation records. Averaging the supplied per-task records recovers all 18 MASS and full-MATE entries in the main comparison to the precision reported in the paper. This is useful support for the reported results, although it does not constitute an independent reproduction.
Critical weaknesses
My main reservation is whether the contribution beyond MASS is sufficiently substantial. MASS already observed that routing-layer preferences vary across tasks. MATE adds margin-based layer selection, score averaging, and routing-guided head selection, but the current comparison does not clearly isolate the benefit of aggregating several layers from that of selecting a better layer using proxy data. The supplementary configurations specify M9 for the two ViT-B backbones and M20 for ViT-L/14 as the single-layer baseline. In eight of the nine settings, these differ from the highest-margin layer computed from the supplied proxy statistics. A higher proxy margin does not guarantee higher classification accuracy, but this makes a single layer selected under the same conditions an important comparison for assessing the central contribution. There is also a discrepancy between the routing-score definition and the implementation. Equation (4) applies softmax directly to negative projection residuals, whereas the supplied code standardizes the scores across tasks before softmax. This changes the score distribution used for thresholding, cross-layer averaging, and head selection. The paper should specify the actual computation used for the reported results.
Questions for authors
How does multi-layer aggregation compare with a single layer selected using the same proxy samples and relative-margin criterion? If this comparison has already been performed, please identify the corresponding results and keep the head-selection rule fixed so that the effect of aggregation is clear.
Were the reported experiments run with the score standardization implemented in router.py? Please reconcile Equation (4) with the implementation and clarify the routing-score computation used in the comparisons.
4 comments
To Reviewer s5Dk (2/2)
W2 & Q2. Routing-score standardization and consistency with Eq. (4)
Thank you for pointing out this discrepancy. Yes. After rechecking the implementation, we confirm that all experiments reported in the original manuscript used the Z-score standardization in router.py, rather than applying softmax directly to raw negative projection residuals.
MATE was developed using the public MASS codebase and inherits the standardization in its topk routing mode. Eq. (4) followed the formulation in the MASS paper but omitted this operation from the actual implementation. We acknowledge that the equation therefore did not fully describe the computation used for the reported results.
Actual routing-score computation
For input , let denote the projection residual for task at layer . The residuals are first standardized across tasks within the current layer:
where and are the mean and standard deviation of the residuals across all tasks for the current input at layer , and is a numerical stability constant.
Each layer then computes its routing scores as
These scores form a soft task distribution at each layer. MATE uniformly averages the distributions over the candidate layer set, as specified in Eq. (9):
Thus, cross-layer aggregation operates on task probabilities computed from standardized residuals, rather than on raw projection residuals from different layers. The aggregated scores are then used for candidate task selection and routing-guided head decision.
Distinction from the relative projection residual margin
Z-score standardization is inherited from MASS and is not a new contribution of MATE. It serves a different purpose from the relative projection residual margin. The latter uses the representation-norm normalization in Eq. (5) to quantify layer-wise task separability for offline candidate layer selection, whereas Z-score standardization is applied when computing online routing scores.
We will correct Eq. (4) and explicitly describe the standardization and routing-score computation in the revised manuscript, ensuring that the mathematical description matches the implementation used in the experiments. The uniform aggregation in Eq. (9) remains unchanged.
Thank you for the detailed response and additional experiments. My main concerns have been addressed. However, the method retains the task-subspace representation and dynamic merging framework used in MASS, with the main changes focusing on layer selection, multi-layer routing aggregation, and classification-head selection. These changes are relatively simple, so my assessment remains borderline between 2 and 3. Taking the additional experiments and revisions into account, I will raise my rating from 2 to 3.
To Reviewer s5Dk
Thank you for your thoughtful reassessment and for raising your rating. We sincerely appreciate your recognition of our additional experiments and revisions. Your feedback has been very helpful in improving our paper.
To Reviewer s5Dk (1/2)
W1 & Q1. Multi-layer aggregation versus proxy-based single-layer selection
Thank you for highlighting the need to distinguish the contribution of multi-layer aggregation from that of proxy-based layer selection. We agree that MASS already observed task-dependent routing-layer preferences. Building on this observation, MATE introduces the relative projection residual margin to quantify layer-wise task separability, selects multiple informative candidate layers, and aggregates their routing scores. The aggregated scores also guide the final classification head decision.
Controlled comparison with a proxy-selected single layer
To isolate the contribution of multi-layer routing aggregation, we conducted an additional comparison of five routing strategies. All variants use the same task models, TSV-M merging of the selected task subspaces, and routing-guided head decision, abbreviated as HG below. Only routing-layer selection and aggregation vary. Proxy-Best Single and MATE Top-5 use the same proxy samples and task-averaged margin Aˉℓ.
Table R1. Average classification accuracy (%) under different candidate layer selection and aggregation strategies. Bold indicates the highest reported accuracy in each configuration.
Specifically:
As shown in Table R1, with routing-guided head decision fixed across all variants, MATE Top-5 outperforms Single-Layer (MASS) in all nine configurations and Proxy-Best Single in seven. These results indicate that MATE’s gains generally extend beyond using proxy data to select a stronger individual routing layer.
Furthermore, Random-5 and All-MLP both outperform Proxy-Best Single in all three ViT-L/14 settings. This provides additional evidence for the value of combining task-discriminative routing information across layers. Meanwhile, MATE Top-5 outperforms both Random-5 and All-MLP in all six ViT-B/32 and ViT-B/16 configurations, supporting the usefulness of margin-based candidate layer selection over these simpler multi-layer alternatives.
The results also show that MATE Top-5 is not the best layer combination in every configuration, particularly on ViT-L/14. The task-averaged margin is therefore a useful separability criterion, rather than a guarantee of the layer combination with the highest final classification accuracy.
Together, these controlled comparisons support the contributions of both candidate layer selection and multi-layer routing aggregation. Multi-layer aggregation uses complementary routing evidence beyond a single proxy-selected layer, while the relative projection residual margin provides a quantitative basis for identifying informative candidate layers.