- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper investigates adapting a pretrained video generator for tactile-pressure estimation from egocentric RGB video. It freezes the Wan2.1-Fun-Control backbone and VAE, trains LoRA adapters, and represents tactile measurements as rendered pressure videos. A multi-stream extension spatially stacks target latents to jointly generate pressure, hand configuration, and hand–object segmentation. Experiments on OpenTouch and EgoTouch evaluate tactile-only and joint prediction, supplemented by pretraining and LoRA-rank ablations and qualitative cross-dataset transfer. The results demonstrate the feasibility of this adaptation, while tactile-only models achieve better tactile reconstruction and contact overlap than the joint models.
Critical strengths
-
A simple adaptation with practical input requirements. The approach requires only RGB video at inference and adapts a frozen backbone through LoRA, without introducing task-specific output heads. This provides a straightforward recipe for exploring vision-to-touch prediction using existing video generators.
-
Evaluation across single-hand and bimanual settings. Testing on both OpenTouch and EgoTouch provides useful evidence that the formulation can accommodate different tactile layouts and output configurations. The comparison between tactile-only and joint prediction also documents a relevant trade-off: generating auxiliary modalities does not automatically improve tactile accuracy.
Critical weaknesses
-
The comparative evidence for the added predictive value remains limited. The paper reports quantitative agreement with recorded tactile signals and appropriately acknowledges that RGB observations do not uniquely determine pressure. However, without a matched pressure-prediction baseline, it remains difficult to assess the benefit of the proposed adaptation relative to simpler supervised predictors or closely related approaches. The HACO comparison evaluates contact localization under different training conditions and does not resolve this issue. In particular, the current comparisons do not establish how much the method improves instance-specific pressure prediction beyond common contact patterns associated with the observed interaction.
-
Several experimental details require clarification to support the conclusions. Appendix A.1 provides a general frozen-backbone, LoRA-training recipe, but its application to the random-initialization control is unclear, as are the corresponding training budget and convergence behavior. The train/test grouping also needs clarification because multiple clips are extracted from individual EgoTouch recordings. Finally, the apparently corresponding OpenTouch joint/rank-64 configurations produce different results in Tables 2 and 6 without explanation. These uncertainties limit the interpretation of the pretraining ablation and the reported generalization results.
Questions for authors
-
Clarification — random initialization Which components are randomly initialized, and which parameters are trainable in this experiment? In particular, is the randomly initialized backbone frozen with only LoRA optimized, or is the backbone trained? Please specify whether the VAE retains pretrained weights and provide the training budget and convergence information needed to interpret the comparison.
-
Evaluation protocol Section 4.2 specifies masked MSE and a threshold of 0.1 for hard IoU. Please clarify the quantity to which this threshold is applied, including how generated RGB renderings are converted into pressure or activity values. What are the exact soft-IoU definition and valid-pixel masks used for evaluation? If evaluation in the original sensor representation is available, how does it relate to the reported rendering-space results?
-
Additional evidence — comparative value Could you provide a matched comparison on at least one dataset against a reasonable supervised pressure-prediction baseline, or identify any existing results that establish the benefit of the proposed adaptation? The comparison should use the same RGB input setting, data split, and evaluation protocol, with any differences in target representation made explicit.
5 comments
Dear Reviewer XS73,
We wanted to briefly follow up on our earlier message, as we have not yet received any further feedback on our rebuttal and additional experiments. With the discussion period ending, we would greatly appreciate it if you could let us know whether our responses have adequately addressed your concerns, or if there are any remaining issues we should clarify.
Thank you
Dear Reviewer XS73,
We want to follow up again on our earlier message as we have yet to receive feedback on the additional experiments and clarifications we provided. We are excited about this work and are very eager to address remaining concerns that you have, while there is still time for discussion.
Thank you, Authors
Dear Reviewer XS73
Thank you once again for your time and constructive feedback on our paper. We have posted our response and supplementary experiments addressing your points regarding method clarification and comparison to additional baselines. As we approach the end of the discussion period, please let us know if any part of our response requires further clarification or if there are remaining questions we can help resolve.
Thank you, The Authors