- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
OTLA improves layer correspondence for transferring task vectors between models with different depths and widths. A relaxed optimal transport problem combines activation similarity and relative depth to obtain soft layer assignments. The same weights combine source activations and task-vector blocks before coordinate alignment. The method uses calibration data and validation-based scaling, without optimizing target-model parameters.
Critical strengths
The contribution is supported by comparisons with reasonable layer-matching alternatives, including Nearest Depth and Kernel matching, and by ablations of the transport design. With BiCo, replacing Nearest Depth with OTLA improves average accuracy in all four transfer directions in Table 1: 87.39→88.59, 87.47→88.35, 81.74→83.84, and 80.42→82.98. The larger gains for deeper-to-shallower transfer make this a useful extension of the alignment pipeline.
Critical weaknesses
The formulation in Eq. (3) is inconsistent with width reduction. The activation dimensions imply , whereas the stated constraint cannot hold when . Algorithm 1 refers to the same equation without specifying a different constraint for this case, leaving the mathematical description of the compression experiments incomplete.
The motivation in Section 3 does not accurately describe BiCo's handling of depth mismatch. It associates BiCo with Block Extension and then describes the strategy as supporting only shallow-to-deep transfer. BiCo already uses nearest-depth layer-index matching and reports deeper-to-shallower transfer in its Section 5.3 and Appendix F.3. The discussion therefore overstates the limitations of the existing approach.
Questions for authors
What constraint and solver are actually used when the source width exceeds the target width, and how does that formulation relate to Equation (3)? Please also clarify whether the Nearest Depth baseline corresponds to the original BiCo layer-matching rule.
2 comments
Thank you for the clarification and the revised manuscript. The revision now specifies the constraints and solution procedures for both expansion and compression. The description of BiCo’s capabilities and its relationship to Nearest Depth matching has also been corrected. These changes address the two concerns I originally raised, and I have no further questions.
We appreciate the reviewer’s recognition of the comprehensive comparisons with reasonable layer-matching alternatives, the supporting transport design ablations, and the consistent improvements across all transfer directions, particularly for deeper-to-shallower transfer. We address the reviewer’s concerns below.
We thank the reviewer for pointing out this inconsistency. The reviewer is correct that the constraint in Eq. (3) applies only when dT≥dS, that is the target width is greater than or equal to the source width. For width reduction (dT<dS), the correct formulation is:
Ri→j⋆=R:R⊤R=IdTargmin∣∣HˉT,j⋆R⊤−HˉS,i⋆∣∣F2
Both formulations (expansion in Eq.(3) and compression as defined above) are solved using the Singular Value Decomposition (SVD) of the cross-covariance matrix (HˉS,i⋆)⊤HˉT,j⋆=U⋆Σ⋆(V⋆)⊤, yielding the closed-form solution is Ri→j⋆=U⋆(V⋆)⊤. The compression formulation above is already used in our implementation, so the reported results are unaffected. We will include the correct equation in the revised manuscript and clarify Algorithm 1 accordingly.
We thank the reviewer bringing this inaccuracy to our attention. Our description was based on the BiCo implementation provided in the merge-and-rebase repository [C], which employs Block Extension for layer matching, as the original BiCo code is not publicly available. As the reviewer correctly points out, BiCo also supports deeper-to-shallower transfer in Appendix F.3. Moreover, the layer-matching rule described in Section 5.3 of the BiCo paper is mathematically equivalent to our Nearest Depth baseline, up to tie-breaking conventions. We will revise the discussion to avoid overstating the limitations of existing approaches and clarify the relationship between our Nearest Depth baseline and the original BiCo matching rule.
[C] Merge-and-Rebase: A Unified Framework and Evaluation Benchmark for Fine-Tuning, Model Merging, and Rebasin, 2026, Panariello et al.