- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies a failure mode in multi-task learning where two causally related tasks are trained with a shared representation. It argues that when an upstream task dominates and task gradients are nearly orthogonal, the stronger task can repeatedly disrupt the weaker downstream task, a phenomenon the authors call orthogonal bullying.
Critical strengths
The paper defines orthogonal bullying as a specific gradient pathology in causally cascaded tasks. However, the paper’s contribution is best viewed as a formalization of a narrow failure mode and a justification for cascade-style architectures, rather than a broadly new multi-task learning paradigm.
Critical weaknesses
- The paper’s central claims are not sufficiently established. It needs to show that causal cascades necessarily lead to the proposed optimization pathology, and that the proposed architecture is necessary rather than just a reasonable cascade-style design. However, the key condition of near-orthogonal gradients is largely assumed or empirically observed, not derived from causal structure. Many theoretical conclusions also depend on strong assumptions, which limits their generality.
- The novelty is somewhat overstated. Second-order curvature causing task interference has prior precedent, and the paper should better distinguish itself from existing gradient-conflict and multi-task methods such as PCGrad, as well as broader modular or causal architectures. 3. The writing is also weak: the abstract is overloaded with details and numbers, while the introduction is unusually short. The theory section contains many formulas and terms but lacks clear explanations. Overall, the scope seems narrow, and the significance of the problem is not convincingly demonstrated beyond the specific setting studied.
Questions for authors
- Can the authors formally derive gradient orthogonality from causal cascade structure, rather than assuming or empirically observing it?
- Can the authors provide broader empirical evidence beyond the narrow metro flow-to-OD setting and synthetic SCMs?