Beyond Gradient Conflict: Multi-Task Landscape-based Optimization via Geometric Independent-task Confluence
Abstract
Multi-task learning (MTL) is largely dominated by gradient-based methods that reconcile conflicting objectives through a single shared update direction. However, under strong task conflict, this shared-trajectory constraint often leads to unstable optimization and degraded performance. We introduce **LOGIC**, **L**andscape-based **O**ptimization via **G**eometric **I**ndependent-task **C**onfluence, a conceptual shift in MTL that replaces a single shared gradient compromise with independently optimized task trajectories that are later coordinated in parameter space. Rather than enforcing agreement at every optimization step, LOGIC allows each task to evolve along its own loss landscape before geometrically aligning task-adapted solutions through a curvature-weighted centroid, with periodic re-anchoring of the task trajectories. By decoupling task-specific optimization from cross-task coordination, LOGIC stabilizes training and enables consistent positive transfer under strong task conflict. Across four benchmarks, three backbones, and both vision and language tasks, LOGIC consistently outperforms thirteen strong gradient-based baselines and frequently surpasses single-task learning, while exhibiting substantially smoother and more stable convergence dynamics.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.