acceptodds
Desk rejected at ICLR 2027

MOMA: Masked Orthogonal Matrix Alignment for Zero-Additional-Parameter Model Merging

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Model merging offers a scalable alternative to multi-task learning but often suffers from substantial performance drops on classification tasks. We attribute this degradation to a geometric misalignment between the merged encoder and static task-specific classifier heads. Existing post-merging methods rely on auxiliary parameters to enforce strict representation alignment, incurring storage and inference overhead that grows with the number of merged models. We challenge this approach by revealing that the misalignment is well-approximated by an orthogonal transformation, rendering such strict alignment unnecessary. Leveraging this insight, we propose MOMA (Masked Orthogonal Matrix Alignment), a plug-and-play post-merging framework that rectifies the misalignment by jointly optimizing a global multi-task vector mask and task-specific orthogonal transformations. Crucially, MOMA absorbs corresponding new parameters directly into existing model weights, achieving state-of-the-art performance among post-merging methods with zero additional parameters and zero added inference cost.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Model Merging, Post Alignment, Classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4BEyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MOMA (Masked Orthogonal Matrix Alignment), a post-merging framework that addresses the misalignment between a merged encoder and task-specific classifier heads. Motivated by empirical analyses suggesting that discrepancies between merged and fine-tuned representations can be approximated by orthogonal transformations, the method jointly learns task-specific alignment matrices and a shared mask over the aggregated task vector. The optimization minimizes representation-level discrepancies with the task-specific fine-tuned models. After training, the mask is incorporated into the encoder weights, and the alignment matrices are absorbed into the classifier heads, introducing no additional parameters or computation at inference time. The authors evaluate MOMA on image and text classification benchmarks and report improved average accuracy over existing post-merging approaches across several merging baselines. The experiments are complemented by analyses of transformation constraints, component contributions, and sensitivity to calibration data availability.

Critical strengths
  • The paper offers a novel perspective by focusing on the misalignment between merged representations and task-specific classifier heads.
  • The empirical results provide convincing evidence of the proposed method’s effectiveness.
Critical weaknesses
  1. The motivation for preserving representation geometry through a strongly constrained transformation is insufficiently justified. The authors compare transformation families with varying degrees of freedom and favor orthogonal transformations because they achieve performance comparable to more flexible alternatives while preserving representation geometry. However, the paper does not adequately explain why geometry preservation should be an objective in addition to predictive performance, or what concrete benefit this preservation provides. Without establishing the importance of this additional objective, the characterization of orthogonal alignment as the best trade-off between performance recovery and geometric preservation is not fully convincing.

  2. The rationale for introducing mask learning is not sufficiently consistent with the earlier motivation for orthogonal alignment. The authors introduce mask learning alongside orthogonal alignment to increase the degrees of freedom in the joint optimization. However, Sections 3.3 and 3.4 argue that a rigid orthogonal transformation is sufficient to correct the representation misalignment while preserving its geometry. This creates an apparent tension: if orthogonal alignment is already sufficient, why is it necessary to modify the encoder itself through a learned mask? Conversely, if mask learning is needed to address discrepancies that a rigid transformation cannot correct, this should be reconciled with the earlier claims. The paper needs a more coherent explanation of why mask learning is necessary and how its role relates to the proposed geometric interpretation.

  3. The advantages of MOMA over Linear + Masking are unclear. Section 5.4 shows that Linear + Masking outperforms MOMA, but the authors describe it as a “relaxed empirical upper bound” rather than treating it as a competing method. Why not adopt this variant instead? The two approaches appear likely to have comparable computational costs, although this requires empirical verification, leaving the practical advantage of choosing MOMA unclear. The results are also consistent with an alternative interpretation: greater optimization freedom and fitting capacity, rather than preservation of representation geometry, may be responsible for the stronger performance. Moreover, the regularization analysis in Appendix D.1, Table 8, reports better performance as the orthogonality regularization is weakened. This observation does not substantiate the benefit of enforcing orthogonality and instead reinforces the need to explain why the orthogonal constraint is preferable.

  4. The claim about prioritizing logit consistency does not clearly reflect the actual optimization objective. Around line 064, the paper states that “MOMA prioritizes output logit consistency by rectifying the inherent orthogonal misalignment.” However, the proposed objective minimizes discrepancies between hidden representations gg, rather than directly optimizing consistency between output logits. The authors should clarify the relationship between the stated goal and the implemented objective, and revise the wording accordingly.

  5. The empirical validation is largely restricted to classification tasks. Validation on natural language generation tasks remains insufficient. Appendix D.3 compares different transformation families and shows that orthogonal alignment achieves representation-level Euclidean distances comparable to those obtained with affine and linear transformations. However, this does not establish the effectiveness of the full MOMA framework for generation. As the authors acknowledge, it remains unclear how directly their conclusions extend to settings with a shared language-model head rather than separate task-specific classifier heads. Furthermore, the parameter-absorption argument is presented for linear heads; its applicability to arbitrary nonlinear heads is not demonstrated. The scope of the claims should therefore be clarified, and stronger validation is needed to support applicability beyond classification.

Questions for authors
  1. Robustness under practical failure modes. Prior work [1] discusses failure modes in more practical model-merging settings. How robust is MOMA under such conditions, particularly when the norms of individual task vectors differ substantially or prediction confidence is low?

  2. Robustness to surrogate calibration data. In practical applications, inputs from each target task may not be available. How robust is MOMA when its mask and alignment transformations are optimized using surrogate data rather than task-specific inputs? How much of the reported performance is retained under this data mismatch?

[1] Yoshida, Kotaro, et al. "Distac: Conditioning task vectors via distillation for robust model merging.", International Conference on Learning Representations, 2026.

5 comments

Thank you for your valuable comments. Our responses are as follows.

W1: The motivation for preserving representation geometry through a strongly constrained transformation is insufficiently justified.

W3: The advantages of MOMA over Linear + Masking are unclear.

Our main contribution is to show that merged representations retain substantial discriminative information and that a restricted, geometry-preserving transformation can correct their mismatch with the classifiers. We agree that a general linear transformation can also be absorbed into a linear classifier head and yields higher accuracy in our experiments. We chose orthogonal alignment to investigate how much of the mismatch can be explained by a simple transformation that preserves distances and angles. Achieving SOTA accuracy was secondary to this goal. Further, orthogonal alignment achieves 88.2% CV accuracy, and relaxing the constraint to allow a general linear transformation raises accuracy to 89.9%. This comparison shows that substantial performance recovery is already possible under the stricter geometric constraint.

W2: The rationale for introducing mask learning is not sufficiently consistent with the earlier motivation for orthogonal alignment.

The two components are complementary: masking adjusts the shared encoder, while orthogonal alignment improves compatibility with task-specific heads. Adding orthogonal alignment to masking improves CV accuracy from 83.5% to 88.2%. This 4.7 percentage point improvement demonstrates a meaningful contribution beyond mask learning and shows that the gains cannot be attributed to masking alone.

W4: The claim about prioritizing logit consistency does not clearly reflect the actual optimization objective.

Our point is that representation drift does not necessarily impair classification by itself. For example, KNN classification on merged representations can outperform some existing merging methods. In the setting studied here, performance degrades when this drift disrupts compatibility between the merged representations and the fixed classifier heads.

Feature-level alignment serves as an effective surrogate objective for logit-level alignment. We have also evaluated direct logit-level alignment using KL divergence, which achieves slightly better performance than our feature-level objective. However, since our main comparison methods, Surgery and ProbSurgery, both perform feature-level alignment, we retain this objective to keep the calibration signal consistent with these baselines.

MethodAccuracy (%)
TA + MOMA (Feature-Level Alignment)88.2
TA + MOMA (Logits-Level Alignment)90.3

W5: The empirical validation is largely restricted to classification tasks.

Our end-to-end experiments cover classification with task-specific linear heads. Appendix D.3 examines representation alignment in generative models, while the effectiveness of the full framework with a shared language-model head and its applicability to nonlinear heads remain open questions. We hope to explore this in future work.

Q1: Robustness under practical failure modes.

Thank you for highlighting the failure modes studied by DisTaC. We conducted additional experiments comparing Surgery and MOMA under the Norm Mismatch and Low Confidence settings. The results below show that MOMA is effective in both settings.

MethodNorm Mismatch (%)Low Confidence (%)
TA + DisTaC70.063.6
TA + Surgery79.677.0
TA + MOMA85.585.1

Q2: Robustness to surrogate calibration data.

Thank you for raising this question. We conducted additional experiments using out-of-distribution calibration data. In the eight-task merging setting, for example, we held out one task at a time for testing, used data from the other seven tasks for calibration, and averaged the results across the held-out tasks.

We observed a substantial drop in accuracy on the CV tasks, from 88.2% to 67.2%, and a smaller drop on the NLP tasks, from 76.3% to 74.4%. We believe one possible reason is that the visual features differ considerably across the CV datasets. For example, some datasets contain images of cars, while others contain images of digits, making transfer across tasks difficult. Although the NLP tasks have different label sets, they share vocabulary and grammatical structures, which may facilitate transfer. We will include this observation in the revised manuscript.

Thank you for your response. However, critical concerns still remain.

First, while the authors now frame the primary goal of the paper as investigating how much of the mismatch can be explained by a transformation that preserves distances and angles, why is exploring this meaningful in the first place? The underlying motivation remains unclear. Furthermore, despite this reframing, the current manuscript still positions MOMA as a practical post-merging framework designed to achieve SOTA performance with zero additional parameters, creating a clear contradiction with the authors' rebuttal.

Second, regarding why mask learning was additionally introduced, the authors only respond to my questions about causality and logical consistency with empirical ablation numbers. Rather, the fact that masking alone yields a +14.4 percentage point improvement, substantially outperforming orthogonal alignment alone (76.4%), undermines the paper's core hypothesis that performance degradation after merging is primarily caused by an orthogonal rotational misalignment. Instead, these results indicate that modifying the feature space itself via encoder mask learning is the dominant factor.

Thank you very much for your careful response and valuable comments. Our reply is as follows.

First, while the authors now frame the

We believe that studying what the mismatch between the merged embedding and the classifier head actually is, and where its limits lie, is meaningful. In particular, when this mismatch is an angle- and norm-preserving orthogonal transformation, we consider this finding itself meaningful. Therefore, we believe the novelty of the orthogonal-transformation finding is greater than that of the SOTA results.

We fully understand your concern about the SOTA results. At the same time, the current best result, MOMA+Linear, is also proposed in our paper and adds no extra parameters. It can be understood as a variant of MOMA. In the next revision, we will revise the wording regarding SOTA and add MOMA-L (the linear version of MOMA) to the main experimental results.

Second, regarding why mask

  1. To strictly guarantee that the transformation is orthogonal, we constrained the orthogonal transformation using torch.nn.utils.parametrizations.orthogonal. This hard constraint is very strict and difficult to converge, resulting in an accuracy of only 76.4. If we instead use an orthogonality regularizer as a soft constraint, the accuracy reaches about 83%, comparable to using masking alone. Given that the orthogonal transformation has far fewer parameters than the mask matrix while achieving comparable performance, we consider the orthogonal transformation to have better parameter efficiency.

  2. We are not claiming that adjusting the feature space is unimportant. In fact, if it can be shown that an orthogonal mismatch between the merged embedding and the classifier head causes the performance degradation, then adjusting the merged embedding, training a transformation, or training the classifier head can all alleviate this orthogonal mismatch. If masking were the sole dominant factor, adding orthogonal alignment after masking should bring no notable improvement. In practice, under the soft constraint, the gain of orthogonality alone (83%) is already comparable to that of masking alone (83.5%), and adding orthogonal alignment on top of masking still improves accuracy by 4.7% (hard constraint, 83.5→88.2) or 5.9% (soft constraint, 83.5→89.4). This indicates that the orthogonal transformation is independently effective.

Thank you again for your suggestions. In the revised version, we will clarify the paper’s contributions in light of our discussion during the rebuttal period, and make the corresponding revisions to the manuscript accordingly.

Reviewer_vyswOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explains the performance degradation following model merging as a misalignment between the merged encoder and classifiers. The authors argue that this discrepancy can be approximated by an orthogonal transformation and propose MOMA, which jointly learns a global task-vector mask and task-specific orthogonal transformations. The learned parameters are absorbed into the existing weights, incurring no additional inference overhead. It outperforms existing post-merging methods across various CV and NLP benchmarks.

Critical strengths
  1. MOMA achieves consistent performance gains across multiple merging methods and CV/NLP benchmarks.

  2. By absorbing the learned transformations and masks into the existing weights, it improves performance without requiring extra parameters or inference costs.

Critical weaknesses
  1. In Table 3, Masking alone (83.5%) outperforms Orthogonal alone (76.4%), and Linear + Masking (89.9%) also surpasses MOMA (88.2%). Furthermore, since a general linear transformation can also be absorbed into the classifier weights, the zero-overhead property is not a unique advantage of orthogonality. Given that prior work utilizing orthogonal geometry [1] already exists, the authors need to clarify the specific advantages offered by orthogonality in MOMA.

  2. SyMerge [2] improves encoder-classifier compatibility by jointly optimizing merging coefficients and task-specific layers. FeatCal [3] also reduces feature drift by directly calibrating existing model weights, requiring no additional inference modules. Although MOMA differs in its use of orthogonal alignment and task-vector masking, its core distinctiveness and tangible advantages over existing studies remain unclear. A direct comparison with them is necessary.

[1] Orthogonal Model Merging, ICML 2026

[2] SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation, ICML 2026

[3] FeatCal: Feature Calibration for Post-Merging Models, NeurIPS 2026

Questions for authors
  1. What specific advantages does orthogonal alignment offer over general linear alignment? In particular, how do you interpret the result where Linear + Masking achieves higher accuracy?

  2. The paper emphasizes logit consistency, but the actual objective optimizes feature-level alignment. Are there any results directly comparing this with logit-level alignment?

4 comments

Thank you for your valuable comments. Our responses are as follows.

W1: The specific advantages of orthogonal alignment over linear alignment and masking are unclear.

Q1: What specific advantages does orthogonal alignment offer over general linear alignment?

We clarify the role of orthogonal alignment from two perspectives.

1. Orthogonal alignment versus linear alignment. Our contribution is to show that merged representations retain substantial discriminative information and that a restricted, geometry-preserving transformation can correct their mismatch with the classifiers. We agree that a general linear transformation can also be absorbed into a linear classifier head and yields higher accuracy in our experiments. We chose orthogonal alignment to investigate how much of the mismatch can be explained by a simple transformation that preserves distances and angles. Achieving SOTA accuracy was secondary to this goal. Further, orthogonal alignment achieves 88.2% CV accuracy, and relaxing the constraint to allow a general linear transformation raises accuracy to 89.9%. This comparison shows that substantial performance recovery is already possible under the stricter geometric constraint.

2. Orthogonal alignment versus masking. The two components are complementary: masking adjusts the shared encoder, while orthogonal alignment improves compatibility with task-specific heads. Adding orthogonal alignment to masking improves CV accuracy from 83.5% to 88.2%. This 4.7 percentage point improvement demonstrates a meaningful contribution beyond mask learning and shows that the gains cannot be attributed to masking alone.

W2: The distinctiveness and advantages of MOMA relative to Orthogonal Model Merging, SyMerge, and FeatCal require clarification and direct comparison.

Thank you for identifying these relevant studies. We agree that the comparison should extend beyond methods that introduce additional inference modules. We will discuss these studies in the revised manuscript.

OrthoMerge merges model weights on the Riemannian manifold of the orthogonal group. It operates during merging, while MOMA calibrates an already merged model. A direct comparison of their results is therefore less informative.

FeatCal is a post-merging calibration method and can be compared directly with MOMA. With 256 calibration samples per task, TA + MOMA achieves 83.5% accuracy, compared with 85.5% reported for TA + FeatCal. MOMA focuses on aligning the merged representations with their task-specific classifier heads. FeatCal calibrates weights layer by layer, which may help it correct feature drift at intermediate layers and could partly explain its higher accuracy in this setting. Combining layerwise calibration with orthogonal alignment is a direction for future work.

MethodAccuracy (%)
TA + FeatCal85.5
TA + Orthogonal + Masking83.5

SyMerge jointly optimizes merging coefficients and a task-specific layer using expert-guided self-labeling. Its combination of adaptive merging and task-specific adaptation makes AdaMerging + MOMA a relevant comparison. On the eight-task ViT-B/32 benchmark, AdaMerging + MOMA achieves 90.7% average accuracy, 0.6 percentage points higher than the result reported for SyMerge.

MethodAccuracy (%)
SyMerge90.1
AdaMerging + MOMA90.7

Q2: The paper emphasizes logit consistency, but the objective optimizes feature-level alignment. How does it compare with logit-level alignment?

Our point is that representation drift does not necessarily impair classification by itself. For example, KNN classification on merged representations can outperform some existing merging methods. In the setting studied here, performance degrades when this drift disrupts compatibility between the merged representations and the fixed classifier heads.

Feature-level alignment serves as an effective surrogate objective for logit-level alignment. We have also evaluated direct logit-level alignment using KL divergence, which achieves slightly better performance than our feature-level objective. However, since our main comparison methods, Surgery and ProbSurgery, both perform feature-level alignment, we retain this objective to keep the calibration signal consistent with these baselines.

MethodAccuracy (%)
TA + MOMA (Feature-Level Alignment)88.2
TA + MOMA (Logits-Level Alignment)90.3

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

Thank you for the detailed response and additional results. My main concerns have been largely addressed. I encourage the authors to incorporate the additional experimental results and comparisons with related work into the revised manuscript, while clarifying the motivation for orthogonal alignment and the choice of feature-level alignment. Please also check the citation formatting throughout the manuscript, using \citep where appropriate. I have updated my rating to 3.

Thank you very much for your careful reading and for increasing your rating. In the final version, we will incorporate the additional experimental results and requested clarifications discussed during the rebuttal, and correct the citation formatting throughout the manuscript.

Reviewer_5M1BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MOMA, a post-merging method that aims to reduce the performance degradation caused by misalignment between merged model representations and task-specific classifier heads. The authors observe that this misalignment can be largely corrected by an orthogonal transformation, without requiring the merged representations to exactly match those of the original fine-tuned models.

Based on this observation, MOMA jointly learns task-specific orthogonal transformations and a global mask over the merged task vector. The learned transformations are absorbed into the classifier weights, while the mask is incorporated into the merged model, avoiding additional parameters or computational overhead at inference time.

Experiments on image and text classification tasks show that MOMA consistently improves several existing model merging methods and generally outperforms previous post-merging approaches. The paper also includes geometric analyses and ablation studies to investigate the proposed method.

Critical strengths
  1. Good empirical results across different merging methods.

The experimental results are convincing overall. MOMA brings substantial improvements to several merging baselines, including Task Arithmetic, Ties, LW AdaMerging, and TSV-M. For example, on ViT-B/32, Ties improves from 72.9% to 90.0% after applying MOMA. The results on different backbones and NLP tasks further support the effectiveness of the method.

  1. A simple and practical approach with no additional inference overhead.

The idea of absorbing the learned transformations into the existing classifier weights is appealing. Unlike Surgery and ProbSurgery, MOMA does not introduce extra parameters at inference time. This makes the method potentially useful in practical model merging scenarios where memory and inference efficiency are important.

Critical weaknesses
  1. The role of orthogonal alignment is not entirely clear.

The paper argues that the representation discrepancy is mainly caused by orthogonal misalignment. However, the ablation results in Table 3 suggest that masking contributes more to the performance improvement than orthogonal alignment alone. On CV tasks, masking improves accuracy from 69.1% to 83.5%, while orthogonal alignment alone reaches 76.4%.

Moreover, Linear + Masking performs slightly better than Orthogonal + Masking (89.9% vs. 88.2%). Since general linear transformations can also be absorbed into linear classifier weights, it is unclear whether orthogonality is essential to the proposed approach or mainly serves as a useful regularization constraint.

I think the method is effective, but the experiments do not fully support the stronger claim that orthogonal misalignment is the main reason for the performance degradation.

  1. Some important implementation details and efficiency considerations are missing.

The method requires unlabeled calibration data and access to the original fine-tuned models during optimization. Although this is consistent with the setting of previous post-merging approaches, the additional training cost and data requirements are not thoroughly evaluated.

In addition, Equation (7) uses a soft orthogonality penalty, while Equation (9) assumes an exact orthogonality constraint. It is not clear how orthogonality is enforced in practice. Interestingly, Appendix D shows that weaker regularization leads to better performance, which raises further questions about how important the orthogonality constraint actually is.

More details on the optimization procedure, computational cost, and sensitivity to calibration data would help strengthen the paper.

Questions for authors
  1. Table 3 shows that masking alone provides a larger improvement than orthogonal alignment, and Linear + Masking performs slightly better than Orthogonal + Masking. Could the authors clarify the specific benefit of enforcing orthogonality? Is the main improvement primarily due to representation alignment or the learned mask?

  2. How is the orthogonality constraint implemented during training? Is it strictly enforced or approximated using regularization? It would be helpful to report the actual orthogonality error of the learned matrices, especially since Appendix D suggests that weaker regularization gives better results.

  3. How sensitive is MOMA to the amount of unlabeled calibration data? Also, could the authors provide a quantitative comparison of training time and memory consumption with Surgery and ProbSurgery? This would help better understand the practical trade-offs of the method.

2 comments

Thank you for your valuable comments. Our responses are as follows.

W1: The role of orthogonal alignment is not entirely clear.

Q1: Table 3 shows that masking alone provides a larger improvement ...

We clarify the role of orthogonal alignment from two perspectives.

1. Orthogonal alignment versus linear alignment. Our contribution is to show that merged representations retain substantial discriminative information and that a restricted, geometry-preserving transformation can correct their mismatch with the classifiers. We agree that a general linear transformation can also be absorbed into a linear classifier head and yields higher accuracy in our experiments. We chose orthogonal alignment to investigate how much of the mismatch can be explained by a simple transformation that preserves distances and angles. Achieving SOTA accuracy was secondary to this goal. Further, orthogonal alignment achieves 88.2% CV accuracy, and relaxing the constraint to allow a general linear transformation raises accuracy to 89.9%. This comparison shows that substantial performance recovery is already possible under the stricter geometric constraint.

2. Orthogonal alignment versus masking. The two components are complementary: masking adjusts the shared encoder, while orthogonal alignment improves compatibility with task-specific heads. Adding orthogonal alignment to masking improves CV accuracy from 83.5% to 88.2%. This 4.7 percentage point improvement demonstrates a meaningful contribution beyond mask learning and shows that the gains cannot be attributed to masking alone.

W2: Some important implementation details and efficiency considerations are missing.

Q2: How is the orthogonality constraint implemented during training?

Q3: How sensitive is MOMA to the amount of unlabeled calibration data?

1. Implementation of orthogonality. The original manuscript did not clearly distinguish the soft constraint used in the geometric analysis from the exact constraint used in MOMA. In Eq. (7), orthogonality is encouraged through a regularization penalty. A smaller penalty weight allows greater deviations from orthogonality and provides more fitting flexibility, consistent with the trend in Appendix D. In Eq. (9), we use torch.nn.utils.parametrizations.orthogonal to enforce orthogonality directly through the parameterization. A soft penalty could alternatively provide additional optimization flexibility, but we retain the exact parameterization to evaluate MOMA under a strict orthogonality constraint. We will explicitly identify which experiments use each implementation in the revised manuscript.

MethodAccuracy (%)
TA + MOMA (torch.nn.utils.parametrizations.orthogonal)88.2
TA + MOMA (Soft Regularization)89.4

2. Calibration data and optimization. MOMA uses unlabeled calibration inputs and the corresponding fine-tuned models as frozen teachers. Only the shared mask and task-specific alignment parameters are optimized. Figure 8 evaluates sensitivity to the amount of calibration data and shows that MOMA generally achieves better performance as more calibration data becomes available.

3. Calibration cost. At inference time, MOMA requires no additional parameter storage or computation, while Surgery requires 131K additional parameters and 33K additional FLOPs per inference, as shown in Table 4. During training, calibration takes 20 minutes for MOMA and 4 minutes for Surgery. Since training is performed only once, we consider the additional 16 minutes acceptable, and this time difference can be offset over repeated inference runs. We will include this quantitative comparison in the revised manuscript and clarify the distinction between calibration cost and inference overhead.

Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.