UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception

Affordance perception aims to localize actionable regions that support embodied interaction, yet 2D and 3D affordance grounding have evolved as separate problems, with different task definitions, supervision formats, datasets, and evaluation protocols. This fragmentation limits the learning of transferable object–affordance semantics across visual and geometric spaces. We propose **Token Router for Tasks**, a general multitask training paradigm for MLLM-based systems that routes contextual hidden states to task-specific branches without requiring the language head to generate predefined task markers. Routed states are supervised directly by branch-specific objectives, enabling dense prediction losses to shape shared MLLM representations. We instantiate this paradigm as **UniAfford**, a unified framework for generalizable 2D–3D affordance perception, together with **UniAfford-Data**, a unified dataset that integrates pixel-level 2D annotations, point-level 3D annotations, and language instructions under a shared object–affordance taxonomy, supporting heterogeneous supervision through semantic-level 2D–3D pairing. UniAfford adopts an MLLM as a shared semantic hub and a modality-aware token router to produce image- and point-cloud-affordance queries. These queries respectively condition a SAM-style pixel decoder and a SONATA-based point decoder, enabling flexible 2D, 3D, and joint affordance inference from image-only, point-cloud-only, or paired multimodal inputs. Extensive experiments demonstrate strong zero-shot generalization across 2D and 3D affordance benchmarks without target-specific fine-tuning, alongside state-of-the-art branch-wise performance under modality-isolated training and evaluation protocols. Ablations demonstrate the importance of token routing, joint 2D–3D supervision, and decoder coupling, while language-head diagnostics show that routed latent states carry meaningful object–affordance semantics. Code and data will be publicly released.