- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The submission argues that successful representation learning lies core of successful multi-task learning (MTL), but that prior work concentrated on the aspect if how MTL aligh with each other or now, while the more important question is whether the resulting representation benefit or not, for which they take the representation variance as a proxy and suggests two additional measures. However, since these terms are not formally defined, and the assumption on e.g. the architecture are not specified, it is hard to judge the merits and correctness of the contribution.
The submission reports on experiments on four rather small-scale MTL setups, where they demonstrate that the new measures reflect different phenomena and can support a filtering mechanism to prevent representation collapse. However, this section as well lacks too much important information to allow proper assessment.
Critical strengths
- The submission revisits a common (but rarely justified) assumption about multi-task learning.
- The submission tries to formally derives their expression and measure from first principles.
Critical weaknesses
-
The manuscript writing is too unclear for publication. It relies on technical background and assumptions about specific MTL architectures and methods without stating them, and it argues about concepts without defining them. The writing style resembles a project report more than a scientific paper, concentrating on a proposed tool and reporting experimental findings rather than putting the work into context of prior work and demonstrating advantages.
-
The impact of the contribution is not made clear. The submission puts forward a measure and suggests this should be reported when evaluating MTL, but the practical benefits this would have remain unclear.
Questions for authors
I do not have any questions that could affect my assessment. The manuscript would need to be completely rewritten, starting with a description of the studied setups, definition of the core quantities and quantitative success criteria. Afterwards, it could undergo a new round of reviews. This is not possible in the form of an author response, so please do not try.
Reviewer–Author Discussion Period Ending Soon
Dear Reviewers,
With the discussion period coming to a close, I encourage you to revisit the authors’ responses and let us know if you have any further comments.
Thank you for your careful reviews and engagement in the discussion!