acceptodds
Withdrawn from ICLR 2027

Bench2Dex: A Bimanual Manipulation Benchmark across Dexterous Hands with Simulated Fingertip Visuo-Tactile Support

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Bimanual dexterous manipulation requires coordinated control of two hands in multi-stage, contact-rich tasks. We introduce Bench2Dex, a simulation benchmark for bimanual manipulation supporting 12 dexterous hands. The benchmark includes 26 tasks involving tool use, interaction with articulated objects, and multi-stage coordination, together with approximately 1.3K human-teleoperated demonstrations. It integrates demonstration collection, synchronized multimodal data, and automated metrics for task completion and intermediate progress within a common framework. Taking a deliberate step beyond vision-only sensing, we equip every supported hand model with simulated vision-based tactile sensing at the fingertips, encoding local contact geometry as image-like tactile observations. Through a unified visuo-tactile interface, we make these observations available alongside external vision and proprioception to support research on tactile representations and multimodal policy learning. We establish reference results using ACT, Diffusion Policy, , and GR00T N1.5 with visual and proprioceptive inputs. We assess robustness across four evaluation channels—None, Equivariance-only, Invariance-only, and Full—covering an unperturbed baseline and controlled combinations of seven scene perturbation types. We further conduct an exploratory GR00T N1.5 comparison with and without tactile input. Averaged across the four evaluation channels, the reported stable-success rates are higher with tactile input on 13 of the 26 tasks and lower on the remaining 13, with task-averaged rates of 27.3% with tactile input versus 28.3% without. By combining a bimanual manipulation benchmark across dexterous hands with simulated visuo-tactile support and initial tactile-conditioned policy evaluations, Bench2Dex provides the community with a common starting point for investigating when tactile information is useful and how manipulation policies can effectively incorporate it.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Dexterous Hands, Simulation Benchmark, Visuo-Tactile

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_B6w4OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Bench2Dex, a simulation benchmark for bimanual dexterous manipulation with cross-embodiment visuo-tactile support. The benchmark includes 26 long-horizon manipulation tasks spanning tool use, articulated-object interaction, and multi-stage coordination, together with 12 dexterous hand embodiments and approximately 1.3K human teleoperated demonstrations. In addition to standard visual and proprioceptive observations, the benchmark provides a unified tactile interface that generates image-like fingertip tactile maps through a surface-aligned ray-casting pipeline over reconstructed hand contact geometries.

Critical strengths
  1. The paper addresses an important and currently underexplored gap in robot-learning benchmarks: unified evaluation of bimanual dexterous manipulation across multiple hand embodiments with visuo-tactile sensing. Existing benchmarks typically focus on parallel-jaw grippers, single embodiments, or vision-only setups, making it difficult to study how tactile representations and manipulation policies transfer across heterogeneous dexterous hands. By integrating 12 hand embodiments, 26 long-horizon tasks, synchronized multimodal observations, and a common tactile interface into a single framework, Bench2Dex provides infrastructure that could become broadly useful for benchmarking cross-embodiment visuo-tactile learning.

  2. The benchmark design and evaluation protocol are relatively comprehensive and thoughtfully structured. In particular, the separation between “equivariance” perturbations (task geometry changes) and “invariance” perturbations (appearance and nuisance variations) provides more diagnostic insight than reporting a single aggregate robustness score. The paper also evaluates several representative policy classes (ACT, Diffusion Policy, π0.5, and GR00T N1.5) and explicitly reports mixed results for tactile conditioning rather than claiming universal gains from touch. This makes the benchmark more credible as an evaluation platform and highlights open research questions instead of overfitting the benchmark narrative to positive tactile results.

Critical weaknesses
  1. The benchmark is entirely simulation-based, including the proposed visuo-tactile sensing pipeline. However, realistic tactile simulation remains an unsolved problem, particularly for deformation, shear, frictional contact transitions, and sensor-specific artifacts that strongly influence dexterous manipulation. The unified visuo-tactile interface relies on surface-aligned ray casting and penetration-depth rendering rather than physically calibrated tactile sensing models. As a result, it is unclear whether conclusions drawn from Bench2Dex about tactile utility, multimodal fusion, or cross-embodiment transfer would remain valid under real-world tactile sensing conditions. This is consequential because the paper’s primary novelty and motivation center on unified visuo-tactile evaluation across embodiments.

  2. The scale of the demonstration dataset appears limited relative to the ambition of the benchmark. Bench2Dex contains approximately 1.3K teleoperated demonstrations distributed across 26 long-horizon bimanual tasks and 12 embodiments, resulting in relatively sparse coverage per task–embodiment setting. Given the complexity of bimanual dexterous manipulation and the current trend toward large-scale robot-learning datasets and foundation-policy training, it is unclear whether this scale is sufficient to support meaningful study of generalizable visuo-tactile learning. The relatively low success rates across all evaluated policies may partly reflect this constrained data regime rather than the intrinsic difficulty of the benchmark itself.

Questions for authors
  1. The benchmark’s main contribution is the unified visuo-tactile interface across dexterous embodiments, but all tactile observations are generated in simulation through penetration-depth rendering and ray casting. Since tactile simulation is still far from solved, especially for friction, deformation, and contact transitions, can the authors discuss how closely these signals reflect real tactile sensors? Any qualitative comparison or evidence of transferability would help assess the practical significance of the tactile benchmark.

  2. Bench2Dex contains about 1.3K demonstrations across 26 long-horizon tasks and 12 embodiments, which seems relatively small given the complexity of bimanual dexterous manipulation. Could the authors clarify the approximate amount of data per task–embodiment pair, and whether they believe current policy performance is mainly limited by benchmark difficulty or by dataset scale?

  3. The tactile-conditioned GR00T experiments show mixed results: tactile input improves performance on some tasks but degrades it on others, with slightly lower average success overall. Do the authors have analysis on what kinds of tasks actually benefit from tactile input? For example, is tactile information mainly useful under occlusion, precise contact alignment, or fine-grained force interaction?

  4. A central motivation of the paper is cross-embodiment visuo-tactile learning, but the experiments mainly evaluate policies within the same embodiment setting. Have the authors tested any cross-embodiment transfer setting, such as training on one subset of hands and evaluating on unseen hands? This would help clarify whether the unified tactile representation provides benefits beyond a standardized data format.

This is still a relatively underexplored area. I would like to raise my score if clearer evidence around tactile realism are offered.

Reviewer_FvTNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents Bench2Dex, a simulation benchmark for visuo-tactile bimanual dexterous manipulation across 12 robot-hand embodiments. The benchmark includes 26 long-horizon tasks involving tool use, articulated-object interaction, and multi-stage bimanual manipulation, together with approximately 1.3K human-teleoperated demonstrations. It provides synchronized RGB-D, geometry-based tactile observations, proprioception, actions, object states, and executable task metrics. To support heterogeneous hand morphologies, the authors introduce a shared surface-aligned tactile interface that reconstructs local contact surfaces and converts geometric penetration depth into image-like tactile maps. The benchmark also includes offline multimodal replay, stage-level progress metrics, efficiency measures, and controlled perturbations for evaluating policy robustness. ACT, Diffusion Policy, π0.5, and GR00T N1.5 are evaluated as representative baselines.

Critical strengths
  1. The benchmark provides unusually broad embodiment coverage. By supporting 12 dexterous hand embodiments, it enables systematic evaluation across heterogeneous hand morphologies rather than restricting analysis to a single robot platform.

  2. The task suite captures challenging long-horizon bimanual behavior. Its 26 tasks include tool use, articulated-object interaction, and multi-stage manipulation, offering a more demanding evaluation setting than benchmarks focused primarily on short-horizon grasping.

  3. The work integrates the full data-generation and evaluation pipeline. Human teleoperation, offline multimodal replay, visual and tactile observations, proprioception, and executable metrics are organized within one framework, which can improve reproducibility and facilitate future extensions.

Critical weaknesses
  1. The effectiveness and physical relevance of the proposed tactile interface are not adequately demonstrated. In Table 3, adding tactile observations improves performance on roughly half of the evaluated tasks, while degrading performance on the remaining tasks, with no consistent overall trend. These results do not establish that the geometry-based tactile observations provide useful information beyond RGB-D, proprioception, or simpler contact representations.
  2. In addition, the paper does not systematically compare alternative ray-casting range, input representations, or fusion strategies. Consequently, the mixed results in Table 3 cannot distinguish whether the limited gains arise from the proposed tactile representation itself, insufficient tactile feature learning, or ineffective multimodal fusion.
  3. The benchmark uses limited domain randomization for key tactile manipulation factors, such as contact friction, tactile noise, sensor delay, and embodiment-specific contact errors, making its robustness and sim-to-real relevance unclear.
  4. The work appears better aligned with the datasets and benchmarks track than applications to robotics, autonomy, planning track, since its main contribution is an integrated benchmark infrastructure rather than a new learning or control method.
  5. It would also benefit from a clearer comparison with UniVTAC, UniDex-ViTac, TactiDex and related dexterous manipulation benchmarks, especially in terms of task coverage, tactile representation, embodiment support, and evaluation protocol.
Questions for authors
  1. How should we interpret the mixed results in Table 3, where adding tactile observations improves approximately half of the tasks but decreases performance on the others? Are these differences caused by the tactile representation, the tactile feature encoder, or the multimodal fusion design?
  2. Does the simulated tactile signal have any quantitative correspondence to real tactile sensors or real-robot measurements? If so, how is this correspondence established?
  3. What advantages does Bench2Dex offer over closely related works such as UniDex, UniDex-ViTac, and TactiDex? In particular, which capabilities or evaluation settings are uniquely enabled by Bench2Dex?
Reviewer_PyBSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Bench2Dex, a benchmark for dexterous manipulation using simulated visuo-tactile sensors. It supports 12 dexterous hands and 26 tasks, with approximately 1.3K human-teleoperated demonstrations. The authors evaluate several popular policy architectures, including ACT, Diffusion Policy, π₀.₅, and GR00T, using the benchmark.

Critical strengths

Unfortunately, I don’t find the paper particularly exciting. I appreciate the effort that must have gone into this work, and I apologize if this sounds harsh. Please see my detailed comments below.

Critical weaknesses

There has been substantial prior work on sim-to-real transfer with visuotactile sensors, as well as a line of benchmark papers, as the authors also discuss. One of the biggest issue is the fidelity of sensor simulation in this work and I would like to know more about it. For example, how well does the simulation capture the characteristics of sensors such as DIGIT and GelSight, particularly for shear sensing? How does it compare with more recent simulators, such as SuperDex?

Given the relatively simplified tactile model, I wonder whether the benchmark sufficiently supports evaluating real-world transferability. It would be helpful to understand whether the findings on this benchmark translate to real-world performance, and if so, what evidence supports this. More broadly, clarifying the benefits of evaluating policies in this simulator, particularly in terms of real-world transfer, would help strengthen the motivation for the benchmark.

Questions for authors

Please address the points above.

Also, it would be great if you can include the dataset statistics in the main text (i.e. the number of demos for each task.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.