Novelty
To the best of the reviewer's knowledge, this work unfortunately may not be the first theoretical work on parameter transfers (line 52-53).
The reviewer found that the analyses appeared to be shared beyond the skeleton of the analyses presented by Meng et al., 2024 and Kou et al., 2023. Moreover, from other communities, feature learning dynamics on transfer learning has been analytically studied in deep linear networks (e.g., Lampinen and Ganguli, 2019; a more recent one, see Tahir et al., 2025). Moreover, a precise asymptotic analysis on perceptron models also revealed similar phenomena (Dhifallah and Lu, 2021). All of these papers include the discussion of negative transfer.
Therefore, the reviewer recommends the authors better contextualize their work within the theoretical research of the topic and highlight the key contributions. Specifically, it would be helpful if the authors could more explicitly indicate the theoretical advancements compared to existing literature (e.g., Meng et al. and Kou et al.) and emphasize the novel insights offered by this model that cannot be captured by perceptron-based models or deep linear networks.
Experiment
The reviewer greatly appreciates authors' efforts into verifying the theory in simulations and connecting it to more common practice. However, the reviewer also wants to raise the worry about how these experiments can support the theory:
First, it seems the parameters chosen for numerical experiments may not satisfy the conditions of 4.1. For example, the dimension d=2000 is noticeably smaller than n2=(100)2 (i.e., the first two settings, while the other settings used a larger n), which further violates the condition 1. Likewise, the number of filters, the signal norm, the learning rates, the initial noise, and the required Γ in Proposition 4.4 seem all fall short of the required conditions of the theory. Thus, the reviewer can only conclude that the experiments can serve as qualitative support for the predicted trend rather than the theorems. Therefore, the reviewer was wondering if the author could further explain, justify, or try to run the experiment on a smaller scale to support the main theorem.
Second, the reviewer interprets the theorem as evaluating not only accuracy but also the phase transition from good to bad transfers. Please correct the following if it is incorrect: The reviewer's reading of the second as well as the third statement in Theorem 4.2 is the transition depends only on dimension d and the term (σp4α2N12∥u∥24+σp4N22∥u+v2∥24)/(σp2α2σp2N1+N2)=:R. If that is the case, could the authors try different configurations of the denominator and numerator with different d and proportion αs and plot the R/d values when the error reaches the same level? This will show whether the R/d ratio stays the same and how close the actual error is to it. Furthermore, could the authors further plot the regressor of R/d to see if the log error is linear? The speed of decay may also indicate how loose the bound is.
Furthermore, the reviewer can understand the potential theoretical convenience for considering correlated signals and/or correlated noise. But as authors hinted (line 159-161), is it possible to numerically verify it in a controlled setting and see, for example, whether negatively correlated signals can impose a stronger negative transfer effect? It would also be helpful to explicitly test on correlated noise (in different degrees) with the other signal direction. In addition, from the formula to the reviewer, it seems in different regimes, practitioners should either do complete parameter transfer or not to do it at all (which seems similar to the results in Dhifallah and Lu, 2021). Could the authors further experimentally verify this idea (and possibly include the correlated scenario, because it could be the case that there is an optimal non-full transfer)?
Finally, regarding to the ViT experiments, the paper documented that "… and its 9th, 10th, and 11th layers are selected as inherited parameters" (line 2537 and 2571). But to the best of reviewer's knowledge, that is quite different to the common practice in transfer learning tasks, where practitioners usually just replace the last layers and fine-tune the model on the downstream task (e.g., Task 2 in the paper). This practice is built on the belief that the early layers learned the common features, while the last layers are related to downstream tasks. So, the reviewer would like to know authors' justification of this decision and how much does this setting link to the common practice. Or, alternatively, the reviewer was wondering whether authors here assumed the feature / knowledge representations distributed equally in the network.
Writing
Unfortunately, the reviewer found the paper difficult to read. Potentially due to the page limit, the proof sketch in the main paper lacks a clear link between the training dynamics of the weights and the two systems, as well as how it links to the final theorems. For example, it is unclear why we need to consider a system in which the solutions are given by the variables xˉtA and xtA. The descriptions of bˉtA,ctA,cˉtA,ctA are also vague in the main text.
The authors provided technical motivations and generic explanations for the conditions and theorems. However, given the number of terms, a more precise technical interpretations of the results would be helpful. These interpretations are welcome to include the reasons for and consequences of the order of some key terms in the bound. Importantly, could the author provide insight into the constant terms, such as C1 and C3, in Theorem 4.2, including if (and how) they are connected?