- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces TouchScale, a 500-hour human visual–tactile dataset containing approximately 87,000 episodes, 1,964 task descriptions, and more than 1,500 objects. Recordings combine head-mounted RGB-D video, two wrist-mounted RGB views, and dense bimanual tactile measurements. The authors evaluate the dataset through cross-dataset tactile prediction, tactile-supervised visual pretraining for action recognition, and mid-training for a robot policy. The reported results include improved zero-shot tactile prediction and an increase in average success rate from 22.5% to 57.5% across four real-robot tasks.
Critical strengths
-
Substantial synchronized data collection. The scale and dense bimanual tactile coverage make the dataset a potentially valuable resource for studying visual–tactile learning.
-
Evaluation beyond a single benchmark. The paper examines tactile prediction, visual representation transfer, and robot policy learning. The real-robot gains are encouraging and make the dataset’s practical relevance more concrete.
Critical weaknesses
-
The dataset’s distinction from closely related work is insufficiently analyzed. Table 1 compares broad statistics, but datasets such as EgoTouch use a notably similar head-camera, dual-wrist-camera, and bimanual-glove setup. HumanTouch is also particularly relevant; TouchSight reports using 500 hours of its recordings. A more direct comparison of capture hardware, actual data diversity, quality, and downstream utility is needed to establish what TouchScale uniquely contributes beyond scale and sensor density.
-
Dataset diversity and quality are not characterized sufficiently to substantiate the headline statistics. The paper reports nine collection settings and over 800 scene configurations, yet many displayed examples appear to use similar tabletops and backgrounds. It does not define what constitutes a distinct scene configuration or report how much the physical backgrounds and work surfaces vary. Likewise, the quality-control procedure is described, but the paper does not report the raw-to-retained data ratio or the proportions assigned PASS, REVIEW, and REJECT. These omissions make it difficult to assess how diverse and usable the 500 hours are in practice.
-
The policy experiments leave the source and scope of the gains unclear. They compare TouchScale mid-training only against skipping that stage. There is no matched mid-training comparison with another visual–tactile dataset, and no evaluation of whether TouchScale improves generalization to new objects, backgrounds, or settings. All policy results come from one robot platform and four tasks. A two-arm task, even using grippers, would additionally help test whether this bimanual egocentric dataset benefits more general manipulation. Such an experiment would probe the value of the visual data; a separate control would still be needed to isolate the contribution of tactile supervision.
Questions for authors
- How is a distinct scene configuration defined? How many distinct physical tabletops, backgrounds, and collection locations are represented, and are any held out during evaluation?
- How many episodes or hours were recorded before quality control? What fractions received PASS, REVIEW, and REJECT, and what fraction was retained in the released dataset?
- Does TouchScale mid-training improve policy success on held-out objects or visual environments? How does it compare with mid-training on another dataset under a matched training budget?
4 comments
Thank you for the detailed rebuttal. I’ve read it carefully and appreciate the experiments you added in response to my review. The comparison with EgoTouch under a matched training budget helps clarify the downstream gains, and the bimanual experiments and vision–touch alignment control strengthen the paper.
My main remaining concern is the environmental diversity of the TouchScale dataset. The evaluation at a separate robot site provides evidence of transfer to a new setting. For the dataset itself, the collection at a single site with four background styles still limits the range of environments represented.
Taking the new results into account, I think a score of 3 remains the best fit for the paper’s overall novelty and the scope of its contributions on this year’s four-level scale. I will keep my original rating.
Thank you for your thoughtful response and for recognizing the value of our additional experiments. We sincerely appreciate your constructive feedback and positive assessment of our work.
Thank you again for your follow-up and for recognizing the value of our additional experiments.
Following your feedback, we have further updated the revised PDF. Sec. 3.2 now explicitly clarifies the single-site collection setup, including five tabletop types, four background styles, and the definition of the 800+ distinct scene configurations. We have also acknowledged environmental diversity as a limitation in Sec. 5. Our next collection round will include more sites and backgrounds.
Therefore, these revisions clarify the environmental diversity of the current dataset and outline our plans to broaden its coverage in future collections. We sincerely appreciate your constructive feedback and support for our work.
Response to igQX
Thank you for the constructive feedback. Your review raises three concerns and three questions:
We hope the responses below address them.
W1. Distinction from EgoTouch and HumanTouch
We agree that EgoTouch and HumanTouch are the closest datasets, and we respectfully note that TouchScale differs from both in tactile density, egocentric depth, and task coverage. HumanTouch reports more recorded hours (1,000+, of which TouchSight trains on 500), so we will remove the claim that TouchScale offers the most recorded hours and add HumanTouch to Table 1.
Task counts follow each dataset's own definition. For downstream utility, TouchScale outperforms EgoTouch at matched hours in zero-shot tactile prediction (0.181 vs. 0.134 cIoU, Table 2) and as robot mid-training data under a matched budget (57.5% vs. 25.0%, W3/Q3). Beyond scale and tactile density, we study how scaling data within TouchScale affects zero-shot tactile prediction and downstream robot manipulation, with fixed sensing hardware and training protocols. As data volume increases, zero-shot cIoU rises from 0.311 to 0.383 and robot success from 30.0% to 57.5% (our response to R2). Thus, TouchScale combines the densest tactile sensing in this comparison with egocentric depth and broad task coverage.
W2/Q1/Q2. Scene diversity and quality control
Thanks for raising this question. TouchScale contains 800+ distinct scene configurations, defined by the object instances and their initial spatial arrangement, including positions, orientations, and relative layout. The same task can have multiple configurations, so these represent different object arrangements, not separate physical environments. Data were collected at one location, with five tabletop types and four background styles. The unseen split in Table 2 holds out entire task descriptions with all of their scene configurations, so it measures transfer to unseen tasks and arrangements on the same tabletops and backgrounds. Transfer to new environments is measured on EgoTactile (Table 2) and at our separate robot site (W3/Q3). Before quality control, we collected approximately 531 hours. Automated screening classified 63.7% PASS, 31.7% REVIEW, and 4.5% REJECT at the episode level, with REVIEW cases further inspected by human reviewers, and we retained 500 hours (94.2%). In conclusion, TouchScale varies tasks, objects, and arrangements at one site and retains 94.2% of the recorded hours. We will include more details into the revision.
W3/Q3. Source and scope of the policy gains
We expanded the policy evaluation as suggested. For the matched mid-training comparison, we compare EgoTouch and TouchScale mid-training under the same model initialization, training budget, 50 robot post-training demonstrations per task, and evaluation protocol. For EgoTouch mid-training, we map its tactile data to the same tactile representation format used in robot post-training.
TouchScale outperforms EgoTouch on all four tasks, by 32.5 points on average (46/80 vs. 20/80, two-sided Fisher exact test, p<0.001). For new objects and environments, our robot site, objects, and scenes are absent from the human data. The same objects appear in the 50 robot demonstrations, so this setting measures transfer from human data to new objects and scenes. For two-arm tasks(Appendix E, Fig. 13), using two xArm6 arms with two BrainCo dexterous hands, we added Bimanual Liquid Transfer and USB Insertion, and TouchScale mid-training improves success from 10% to 30% and from 0% to 30%, respectively, over 20 trials per task (12/40 vs. 2/40 pooled, two-sided Fisher exact test, p=0.006). For a control that isolates tactile supervision, we randomly re-pair visual and tactile observations during mid-training while keeping the training objective, underlying data, training budget, and robot post-training procedure unchanged. Average robot success drops from 57.5% with aligned pairs to 28.75% with randomly re-paired pairs (46/80 vs. 23/80, two-sided Fisher exact test, p<0.001), close to the 22.5% without mid-training (p=0.47). Therefore, TouchScale beats EgoTouch under a matched budget, extends to dual-arm tasks, and helps only when vision and touch stay aligned.