DexOmni: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation

Dexterous manipulation requires reliable contact establishment, temporal coordination of multi-finger and bimanual actions, as well as maintenance of achieved goals, while also transferring these abilities to unfamiliar objects, interactive relationships, and skill combinations. Herein, we introduce DexOmni, a simulation data platform and benchmark featuring 66 tasks whose feasibility was verified through human demonstrations, 18,000 curated human demonstrations, high-fidelity re-rendering, and visual and goal-preserving prompt randomization. Covering a diverse range of interaction types, task structures, and asset types, core-50 provides demonstrations and execution tests for 50 tasks spanning functional tool use, numerical and spatial grounding, articulation, and bimanual manipulation. Transfer-16 reserves 16 tasks to evaluate asset and relation transfer, skill composition, and deformable manipulation without target-task demonstrations in the benchmark training set. We compare task-specialized diffusion policies, vision-language-action models, and world-action models using success and native-stage diagnostics. Across 40,000 Core-50 episodes, the best of eight multi-task checkpoints reaches 29.1% task-macro success. Success rates are 35–51% for pushing, articulation, and pick-and-place but only 2–3% for tool use and precision fitting; this gap persists after controlling for demonstration count and duration. Failures in contact establishment, temporal coordination, and goal maintenance emerge as the main bottlenecks in multi-finger and bimanual control. Transfer-16 leaves the most headroom; ordered progress separates its checkpoints, and compositions add another failure: policies repeat finished steps or start later ones early. Overall, DexOmni provides a challenging benchmark for systematically evaluating the reliability and generalization of embodied policies for dexterous robotic hands.