acceptodds
Desk rejected at ICLR 2027

DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present DONIC, a generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction. Reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavior cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one policy to learn coordinated manipulation across heterogeneous references. Without fine-tuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, exceeding the stronger of two bimanual tracking baselines by 77.36 and 45.73 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generalist Policy, Real2Sim, Hand Object Interaction

Related papers

Discussion (0)

Sign in to comment.