- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes an "endpoint-selection framework" to analyze fine-tuning. It argues that narrow support fails to uniquely determine a model, and different parameterizations select different interpolating endpoints. The authors provide theoretical derivations for linear and signed-square factorized coordinates, and conduct empirical experiments
Critical strengths
None
Critical weaknesses
-
Unacceptable writing quality There are too many jargons such as "training fiber" "endpoint selector", "witness-specific equivalence", and these notions are trivially linear algebra items. Besides, all the theorems look separated, I do not understand how they are connected to a coherent story. The paper's presentation is disproportionately complicated relative to its underlying mathematical content.
-
The theoretical contributions are trivial and heavily overclaimed.
The core theoretical result (e.g., Theorem 6.3, "support-identifiability collapse") is merely the fundamental theorem of linear algebra (row-space/null-space decomposition). Standard minimum-norm interpolants and linear right-inverses are repackaged with new terminology ("parameterization-conditioned endpoint selection"). Claiming this as a novel, profound theory is unacceptable.
Questions for authors
- How do these theorems connected to each other and your empirical observations?
- What is the main nontrivial theoretical contribution beyond existing implicit-bias results, and which experiment actually validates that contribution in a nonlinear Transformer?