acceptodds
Under review as a conference paper at ICLR 2027

Structure Follows Causality: Causally-Aligned Representation Separation Breaks Orthogonal Bullying in Multi-Task Learning

Abstract

When tasks form a causal cascade in multi-task learning—the upstream output conditions the downstream task—one shared representation must encode both the causal parent mechanism and the causal child mechanism, which in the regime we characterize (SGD-class optimizers, sparse high-dimensional downstream, incoherent coupling matrices) occupy incompatible subspaces: task gradients are near-orthogonal (, ) and wildly imbalanced (), so the dominant task's updates repeatedly erase the weaker task's learning as the latter approaches its optimum—whose pure-curvature form () we verify constructively as a transient rather than asymptotic window. We call this orthogonal bullying and prove it is structural, not algorithmic: the flat joint objective has no stationary point in the persistent excitation region, bullying is unavoidable under testable conditions on dominance, curvature coupling and the crossover threshold, and post-hoc mediators face a detection blind spot or a mediation trilemma (Thms. 1, 2, 3a/3b). We advance Structure Follows Causality—the computational graph must mirror the task causal graph—realized by the Causal Graph Architecture (CGA): each DAG node lives in its own module and conflicting gradient paths are cut by stop-gradient (Props. 4/4, edgewise on any DAG; one line of .detach()), compiled deterministically from any causal DAG (Alg. 1). Under a symmetric comparison protocol—the flat baseline is jointly trained and must satisfy a protection constraint that 's loss never increases at any step, without which flat fails outright and the rate comparison is vacuous—CGA reduces the iteration complexity of reaching upstream accuracy from to (Thm. 5); a cascade-gap decomposition (Thm. 6) shows CGA shares the flat model's information-theoretic floor—its value is representation separation of causal mechanisms, not a spurious statistical gain. On controlled synthetic SCMs CGA holds the single-task floor with near-zero variance (T2 0.10–0.12, T1 0.13; std0.003; 12 configs, 3 seeds), while Flat fails in mechanism-specific segments; on the v2 legacy -grid CAGrad's upstream error crashes to 71.9–87.0% and GradNorm's to 53.3% (Flat and CGA stay 1.1%). Under strong coupling the system enters a first-order coupling protection regime where bullying does not activate, bounding the assumptions' scope; Flat still pays a 1.7–2.1 tax and CGA matches the floor. On a production 270-station metro system (Adam + global clipping), joint-training logs record the signature—gradient cosine pinned at 0.00, –, failed mediation, WAPE 36.6%37.7%—registered as motivational log only; not a mechanism check, since the production Adam+clipping setting is not reproduced by either controlled arm; the separated flow predictor reaches 10.34% WAPE (Stage-1 trained alone). A pre-registered three-arm exploratory comparison on real large-scale OD data under an identical anti-collapse recipe (single seed; multi-seed confirmation pending) gives flat joint 62.1%, single-task OD without flow 61.2%, CGA 42.7%—a 19.4pp gap indicating joint training cancels the parent's information contribution, preliminary evidence that only causal structure converts it into realizable gain. Further controlled experiments close three gaps: pure-curvature erasure is realized constructively (81/81 single-step, 27/81 sustained) but is transient and unreachable in the MLP family; eight alternative flat protocols all fail to reproduce CGA's floor; and structured sharing (MMoE/PLE/AdaShare) relieves the downstream only by transferring cost upstream (T1 0.300 vs CGA 0.13), leaving CGA the only method on the floor for both tasks. We also give a three-part decision procedure (directionality bullying indicators). Boundaries and negative results are reported honestly: the controlled stays below production scale, and the erasure window is transient.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
multi-task learning, gradient conflict, orthogonal bullying, causal representation learning, stop-gradient, convergence analysis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jpnjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies a failure mode in multi-task learning where two causally related tasks are trained with a shared representation. It argues that when an upstream task dominates and task gradients are nearly orthogonal, the stronger task can repeatedly disrupt the weaker downstream task, a phenomenon the authors call orthogonal bullying.

Critical strengths

The paper defines orthogonal bullying as a specific gradient pathology in causally cascaded tasks. However, the paper’s contribution is best viewed as a formalization of a narrow failure mode and a justification for cascade-style architectures, rather than a broadly new multi-task learning paradigm.

Critical weaknesses
  1. The paper’s central claims are not sufficiently established. It needs to show that causal cascades necessarily lead to the proposed optimization pathology, and that the proposed architecture is necessary rather than just a reasonable cascade-style design. However, the key condition of near-orthogonal gradients is largely assumed or empirically observed, not derived from causal structure. Many theoretical conclusions also depend on strong assumptions, which limits their generality.
  2. The novelty is somewhat overstated. Second-order curvature causing task interference has prior precedent, and the paper should better distinguish itself from existing gradient-conflict and multi-task methods such as PCGrad, as well as broader modular or causal architectures. 3. The writing is also weak: the abstract is overloaded with details and numbers, while the introduction is unusually short. The theory section contains many formulas and terms but lacks clear explanations. Overall, the scope seems narrow, and the significance of the problem is not convincingly demonstrated beyond the specific setting studied.
Questions for authors
  1. Can the authors formally derive gradient orthogonality from causal cascade structure, rather than assuming or empirically observing it?
  2. Can the authors provide broader empirical evidence beyond the narrow metro flow-to-OD setting and synthetic SCMs?
Reviewer_4pwXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper argues that when one task’s output conditions another in multi-task learning, a shared representation suffers “orthogonal bullying”: the task gradients are nearly orthogonal and very imbalanced in size, so the dominant task’s updates repeatedly erase the weaker task’s progress through second-order curvature damage. To the best of my understanding, the authors prove that this flat setup has no stationary point, that bullying is unavoidable past a measurable threshold, and that post-hoc gradient mediators either never trigger or must sacrifice one of the tasks. The authors propose giving each task its own module and blocking interfering gradients between them, which lets the model train faster without lowering the best achievable error. In experiments, this approach consistently matched the best possible performance, while standard shared models and competing fixes fell short. The real-world results look promising, but the authors admit the evidence is still limited.

Critical strengths
  • Theorem 6 frames the contribution: it proves that cascading cannot lower the statistical floor. A stage-2 model is still a function of X, so any gain must come from optimization.
Critical weaknesses
  • The manuscript reads like largely unedited LLM-generated text. It contains revision-history artifacts (“key correction,” “this (third) version,” “honesty record,” “moved from the main text,” “withdrawn”), self-referential hedging (“Honest reading,” “honesty declaration”), and internal inconsistencies of a kind that careful author revision would normally catch. Whatever the drafting process, the result has not been edited into a coherent paper. I believe this should have been desk rejected.

  • The paper is very hard to read: the abstract is far too long, revision history is left in the paper, notation is overloaded, figures are not annotated in English, metrics change across experiments, nearly every claim comes with a caveat.

Questions for authors
  • Is the mechanism actually operating in your MLP experiments? You report that pure-curvature erasure (E > 1) never occurs in the MLP family, and the main-grid median is only 2–10 (Table 7). Could you compute the crossover threshold for each cell of Table 1 and report the fraction of training steps where main-grid median > crossover threshold and E > 1? If failures happen mostly where bullying is inactive, what explains them, and why attribute them to orthogonal bullying rather than, say, seed-dependent wandering or step-size limits?
Reviewer_h6UjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the "orthogonal bullying", which means that, although the task gradients are near-orthogonal, the update of one task can still affect another task. They provide theoretical analysis on the reason behind it. Moreover, the authors find that "orthogonal bullying" can be overcome with stop-gradient. They provide theoretical and empirical evidence to support their claims.

Critical strengths
  • "orthogonal bullying" is an interesting phenomenon. This phenomenon is meaningful and valuable.
  • The authors show that "orthogonal bullying" can be overcome with stop-gradient, which sounds reasonable. The experiments verify the effectiveness of stop-gradient.
Critical weaknesses
  • The theoretical results are built on informal assumptions and conditions. Assumption 1 should formally define the condition cos⁡θ≈0\cos\theta \approx 0. Based on the remark, the authors seem to mean ∣cos⁡θ∣≤ϵ|\cos\theta| \leq \epsilon for some parameter ϵ\epsilon. In Theorem 1, the condition ϵ≪1\epsilon \ll 1 should be used precisely as well. These informal statements are not appropriate in formal theorems; the authors should specify an explicit constant or an asymptotic order. The same issues also appear in L1≫L2L_1 \gg L_2, κg≫1\kappa_g \gg 1
  • The organization and presentation of the paper make it difficult for me to follow. The abstract is too long and contains too many details, making it difficult to quickly understand the main contributions. The introduction provides insufficient context and motivation. The related work section briefly lists the existing methods but does not carefully discuss them. Theoretical results are listed informal. Explanations and empirical measurements are placed inside the theorem statements in Theorem 1 and Theorem 3a. These remarks should be separated from the formal claims. Moreover, abbreviations such as “OD” and “SCM” are used in the abstract before being explained. Expressions such as “rate tax” and “residual tax” are also insufficiently explained.
Questions for authors
  • In Theorem 5, the authors require the constraint "T2T_2’s loss never increases" while convexity is assumed only for CGA. Are these conditions fair for comparison?
  • Can you provide formal conditions for cos⁡θg≈0,ϵ≪1,L1≫L2,κg≫1\cos \theta_g \approx 0, \epsilon \ll 1, L_1 \gg L_2, \kappa_g \gg 1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.