- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces TacWM, a unified multimodal world model for dexterous manipulation that jointly models visual observations, tactile feedback, robot actions, and object poses using a diffusion transformer.
Critical strengths
Unlike action-only policies or vision-and-touch-only models, TacWM feeds future object poses directly into prediction. Modeling object motion, contact response, visual change, and actions together gives a principled way to capture the coupled dynamics of dexterous manipulation.
Its cross-embodiment alignment tackles a real barrier to scaling multimodal manipulation models across platforms. Heterogeneous pretraining data helps, and the full model posts the best average score across tested tasks. Modality ablations further back the multimodal design.
Critical weaknesses
TacWM posts a 36.5% average success rate, just 2.8 points above the strongest baseline at 33.7%. It wins on several tasks, but Sliding reaches only 86.0%, well short of π0.5's 97.0%.
The paper also doesn't report variance across training seeds or confidence intervals, so it's unclear whether the gains are statistically real.
Despite jointly predicting future vision, touch, actions, and object poses, the paper only quantifies tactile prediction, and only against a persistence baseline. It skips systematic checks on future video quality, object-pose accuracy, and cross-modal physical consistency.
Cross-embodiment tests also only cover platforms already seen in training. There's no evidence yet for generalization to unseen embodiments or new tactile sensor setups.
Questions for authors
see weakness