AFUN: Towards an Affordance Foundation Model for Functional Understanding

Affordance understanding helps connect what a robot sees with what it should do. It provides an interpretable way for robots to manipulate objects in open, messy real-world environments. However, building a general affordance model is still difficult. Such a model needs to understand both where to interact with an object and how the object should move after interaction. It also needs to work across many different environments, objects, and tasks. Most existing methods solve only part of this problem. Some methods can find the relevant object region but do not predict a usable motion. Others can predict motion, but they do not scale well to diverse real-world settings. In this paper, we introduce AFUN, a step toward a general affordance foundation model for understanding object functionality. Given one RGB-D image and a language instruction, AFUN predicts a task-specific functional mask, showing where to interact, and a 3D motion curve, showing how the object should move after contact. To improve generalization, we build a large-scale data pipeline that converts data from robots, humans, simulations, and real-world scans into a unified affordance format. This format includes language instructions, functional masks, and object-centered 3D motion labels. We evaluate AFUN in three ways. For affordance segmentation, AFUN outperforms all baselines by a large margin across 8 test sets from 4 benchmarks, improving mean gIoU/cIoU by +23.9/+26.3. For contact-point prediction, it predicts much more accurate contact points, improving hit rate by 12.7-61.3% over the best baseline. For 3D motion prediction, it achieves the best results on all three test sets. Finally, AFUN can be used for real-world robot manipulation without robot-specific finetuning or task-specific heuristics, showing its ability to adapt to open-world affordance tasks.