DexBeyond: Scaling Dexterous Manipulation Through Iterative Hand-Object Interaction Generation

Learning dexterous manipulation depends on continuous hand-object interaction data that capture fine-grained finger coordination. Many existing datasets emphasize grasping and placement, with limited coverage of both in-hand motion diversity and object variety. We present $\text{DexBeyond}$, a unified framework that combines a large-scale hand-object interaction dataset, a generative expansion model, and an iterative data flywheel to scale dexterous hand-object interaction data. We first construct $\text{DexBeyond-D}$, a large-scale hand-object interaction dataset comprising 20,891 sequences and 58.03 hours of valid hand-object interaction. It covers 219 object instances from 77 object categories and 30 manipulation behaviors, and provides synchronized 3D hand and object poses with the corresponding object meshes. Because exhaustive real-world capture is impractical, we develop $\text{DexBeyond-G}$ to expand interaction diversity for captured objects and augment the dataset with generated motions, thereby improving downstream controller performance. Although DexBeyond-G can propose interactions for unseen objects by replacing its mesh condition, the resulting motions often exhibit penetration, lost contacts, and other physical inconsistencies. We therefore introduce $\text{DexBeyond-FW}$, an iterative, self-improving data flywheel that turns these proposals into feasible interaction data, enabling generalization to unseen objects without additional real-world collection. Experiments demonstrate that DexBeyond scales interaction synthesis across motions and objects, with downstream gains in simulation and the real world and progressive performance improvements on unseen objects.