- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Learning Transferable Directions (LTD), which performs transfer at the level of input-dependent functional changes induced by source blocks. LTD estimates a target descent direction, shortlists candidate source blocks, and combines their functional directions through sparse optimization. Experiments cover vision, domain shift, and language settings, with generally positive results.
Critical strengths
The idea of measuring transferability through local functional effects rather than global parameter similarity is interesting. The experimental evaluation is also reasonably broad and includes useful ablations.
Critical weaknesses
The main motivation is not sufficiently clear. In particular, I found it difficult to understand why LTD is fundamentally preferable to simpler input-dependent routing over source components. The paper introduces substantial machinery around a relatively simple intuition, including direction-field estimation, proposal learning, sparse optimization, and several selection stages, without convincingly justifying this complexity. More generally, the presentation is difficult to follow and the distinction from related routing and expert-combination approaches remains unclear.
Questions for authors
-
Can the authors provide a simple setting where standard input-dependent routing fails but LTD succeeds specifically because of functional-direction alignment?
-
How sensitive is LTD to the choice of source-block partition and fusion layer?
-
What is the full target-adaptation computational cost, including direction computation, cross-fitting, proposal training, and model selection?