acceptodds
Under review as a conference paper at ICLR 2027

Beyond Gradient Conflict: Multi-Task Landscape-based Optimization via Geometric Independent-task Confluence

Abstract

Multi-task learning (MTL) is largely dominated by gradient-based methods that reconcile conflicting objectives through a single shared update direction. However, under strong task conflict, this shared-trajectory constraint often leads to unstable optimization and degraded performance. We introduce **LOGIC**, **L**andscape-based **O**ptimization via **G**eometric **I**ndependent-task **C**onfluence, a conceptual shift in MTL that replaces a single shared gradient compromise with independently optimized task trajectories that are later coordinated in parameter space. Rather than enforcing agreement at every optimization step, LOGIC allows each task to evolve along its own loss landscape before geometrically aligning task-adapted solutions through a curvature-weighted centroid, with periodic re-anchoring of the task trajectories. By decoupling task-specific optimization from cross-task coordination, LOGIC stabilizes training and enables consistent positive transfer under strong task conflict. Across four benchmarks, three backbones, and both vision and language tasks, LOGIC consistently outperforms thirteen strong gradient-based baselines and frequently surpasses single-task learning, while exhibiting substantially smoother and more stable convergence dynamics.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.