- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Bench2Dex, a simulation benchmark for bimanual dexterous manipulation with cross-embodiment visuo-tactile support. The benchmark includes 26 long-horizon manipulation tasks spanning tool use, articulated-object interaction, and multi-stage coordination, together with 12 dexterous hand embodiments and approximately 1.3K human teleoperated demonstrations. In addition to standard visual and proprioceptive observations, the benchmark provides a unified tactile interface that generates image-like fingertip tactile maps through a surface-aligned ray-casting pipeline over reconstructed hand contact geometries.
Critical strengths
-
The paper addresses an important and currently underexplored gap in robot-learning benchmarks: unified evaluation of bimanual dexterous manipulation across multiple hand embodiments with visuo-tactile sensing. Existing benchmarks typically focus on parallel-jaw grippers, single embodiments, or vision-only setups, making it difficult to study how tactile representations and manipulation policies transfer across heterogeneous dexterous hands. By integrating 12 hand embodiments, 26 long-horizon tasks, synchronized multimodal observations, and a common tactile interface into a single framework, Bench2Dex provides infrastructure that could become broadly useful for benchmarking cross-embodiment visuo-tactile learning.
-
The benchmark design and evaluation protocol are relatively comprehensive and thoughtfully structured. In particular, the separation between “equivariance” perturbations (task geometry changes) and “invariance” perturbations (appearance and nuisance variations) provides more diagnostic insight than reporting a single aggregate robustness score. The paper also evaluates several representative policy classes (ACT, Diffusion Policy, π0.5, and GR00T N1.5) and explicitly reports mixed results for tactile conditioning rather than claiming universal gains from touch. This makes the benchmark more credible as an evaluation platform and highlights open research questions instead of overfitting the benchmark narrative to positive tactile results.
Critical weaknesses
-
The benchmark is entirely simulation-based, including the proposed visuo-tactile sensing pipeline. However, realistic tactile simulation remains an unsolved problem, particularly for deformation, shear, frictional contact transitions, and sensor-specific artifacts that strongly influence dexterous manipulation. The unified visuo-tactile interface relies on surface-aligned ray casting and penetration-depth rendering rather than physically calibrated tactile sensing models. As a result, it is unclear whether conclusions drawn from Bench2Dex about tactile utility, multimodal fusion, or cross-embodiment transfer would remain valid under real-world tactile sensing conditions. This is consequential because the paper’s primary novelty and motivation center on unified visuo-tactile evaluation across embodiments.
-
The scale of the demonstration dataset appears limited relative to the ambition of the benchmark. Bench2Dex contains approximately 1.3K teleoperated demonstrations distributed across 26 long-horizon bimanual tasks and 12 embodiments, resulting in relatively sparse coverage per task–embodiment setting. Given the complexity of bimanual dexterous manipulation and the current trend toward large-scale robot-learning datasets and foundation-policy training, it is unclear whether this scale is sufficient to support meaningful study of generalizable visuo-tactile learning. The relatively low success rates across all evaluated policies may partly reflect this constrained data regime rather than the intrinsic difficulty of the benchmark itself.
Questions for authors
-
The benchmark’s main contribution is the unified visuo-tactile interface across dexterous embodiments, but all tactile observations are generated in simulation through penetration-depth rendering and ray casting. Since tactile simulation is still far from solved, especially for friction, deformation, and contact transitions, can the authors discuss how closely these signals reflect real tactile sensors? Any qualitative comparison or evidence of transferability would help assess the practical significance of the tactile benchmark.
-
Bench2Dex contains about 1.3K demonstrations across 26 long-horizon tasks and 12 embodiments, which seems relatively small given the complexity of bimanual dexterous manipulation. Could the authors clarify the approximate amount of data per task–embodiment pair, and whether they believe current policy performance is mainly limited by benchmark difficulty or by dataset scale?
-
The tactile-conditioned GR00T experiments show mixed results: tactile input improves performance on some tasks but degrades it on others, with slightly lower average success overall. Do the authors have analysis on what kinds of tasks actually benefit from tactile input? For example, is tactile information mainly useful under occlusion, precise contact alignment, or fine-grained force interaction?
-
A central motivation of the paper is cross-embodiment visuo-tactile learning, but the experiments mainly evaluate policies within the same embodiment setting. Have the authors tested any cross-embodiment transfer setting, such as training on one subset of hands and evaluating on unseen hands? This would help clarify whether the unified tactile representation provides benefits beyond a standardized data format.
This is still a relatively underexplored area. I would like to raise my score if clearer evidence around tactile realism are offered.