DexSimWAM: Scaling Dexterous World Action Models with Verified Simulation Data

Learning transferable dexterous manipulation policies is constrained by the scarcity of large-scale robot data. Simulation offers scale, but synthesizing dexterous trajectories is substantially harder than for grippers: the data must cover diverse multi-finger contacts and hand–object configurations while satisfying language-specified grasp and motion semantics. We introduce DexSimWAM, which combines an agent-guided, retrieval-based synthetic-data pipeline with large-scale World Action Model (WAM) pre-training and task-specific post-training. Given a language-specified task and verifier feedback, the agent guides scene sampling, grasp retrieval, optimization-based path planning, and bounded expansion of a physically validated object-frame grasp bank. Deterministic filters and fixed whole-task verification yield a 16k-hour corpus spanning diverse objects, scenes, grasp styles, and unilateral and bimanual behaviors. On this corpus, DexSimWAM jointly pre-trains visual-dynamics prediction and synchronized arm–hand action generation. Task-specific post-training then adapts the shared WAM with scalable simulation data and targeted real-robot trajectories, supporting 6D object reorientation, functional pouring, articulated-object interaction, long-horizon semantic rearrangement, direct sim-to-real deployment, and physical-domain refinement. Project website: https://dexsimwam-anonymous.github.io/.