MoDiTac: Modality-Decoupled Diffusion Transformer for Preference-Controlled Tactile Generation

Tactile data capture local object deformation and contact properties, but collecting them requires specialized sensors and physical interaction, making large-scale acquisition more difficult than for visual data. Visual-to-tactile generation can alleviate this bottleneck by providing scalable synthetic data for tactile models. Existing generators primarily emphasize paired reconstruction quality, while paying less attention to whether the generated tactile data suit different downstream tasks. Diffusion models optimized with mean squared error tend toward conservative mean predictions, which can suppress the high-frequency textures that describe fine-grained surface deformation. Tactile detail recovery is further complicated by the mismatch between scene-level visual context and local tactile response, as well as conflicts between heterogeneous representations. MoDiTac addresses these challenges within a unified model. Tactile-guided visual selection extracts tactile-relevant visual evidence from the full scene according to the current generative state. Modality-decoupled projections reduce interference between visual and tactile representations before joint self-attention. Preference-controlled global-context injection maps a continuous preference coordinate to different reconstruction–perception operating points, allowing one model to provide tactile data for different task requirements. Experiments on multiple visual–tactile datasets show that MoDiTac improves both reconstruction and perceptual quality. Material recognition and Press–Sliding recognition favor different regions of the preference continuum, indicating that no single output is uniformly optimal across tactile tasks. MoDiTac provides an explicit interface for selecting task-appropriate tactile data.