- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Tactile–visual–language models may encounter changes in modality reliability due to variations in lighting, contact quality, or sensor hardware. Less reliable modalities may even exhibit higher prediction confidence, thereby degrading fusion performance. To address this issue, this paper proposes Alignment Trust, which uses class text embeddings as semantic references to estimate the relative reliability of visual and tactile modalities and reduce the influence of unreliable modalities on classification. Furthermore, the authors introduce Anchor Realignment to learn a shared feature transformation matrix (R) by aligning fused class-level features with their corresponding text anchors and refining their class discriminability. The learned transformation is then applied to both visual and tactile features to improve class separation. Experiments on two pretrained models, UniTouch and AnyTouch, across four visual-tactile recognition benchmarks demonstrate the effectiveness of the proposed method.
Critical strengths
(1) Practical Problem. The paper addresses a practical problem in visual-tactile recognition: unreliable modalities may remain overconfident under changing sensing conditions, thereby degrading multimodal fusion performance. By leveraging shared text embeddings as semantic references, the proposed Alignment Trust and Anchor Realignment addresses two complementary challenges: how to assign modality weights based on their reliability and how to improve the class discriminability of feature representations.
(2) Comprehensive Results. The experiments cover two pretrained models, four recognition datasets or benchmarks, and both natural distribution shifts and visual, tactile, and mixed corruptions, enabling evaluation under diverse modality reliability conditions. Furthermore, the ablation studies validate the effectiveness of both Alignment Trust and Anchor Realignment.
Critical weaknesses
(1) Novelty compared with Existing Works. One of my main concerns is that the essential differences between the proposed method and existing approaches for modality reliability estimation and feature alignment remain unclear. Alignment Trust (Eqs. (3)–(5)), which estimates modality reliability using historical statistics and reference anchors to adjust fusion weights, shares a similar idea with the Anchor-based Reliability (Eqs. (10)–(13)) and the Reliability-aware Gate Prior (Eqs. (18)–(20) and (22)) in MG-MTTA [1]. Furthermore, Anchor Realignment (Eqs. (6)–(7)), which employs orthogonal Procrustes to align modality features with text anchors, is conceptually similar to the unsupervised feature-to-text prototype alignment in U-LFA [2] (Eq. (7), with the orthogonal Procrustes formulation in Eqs. (3)–(4)). The authors should further clarify the key technical differences between these methods and the proposed approach. It would also be beneficial to provide experimental comparisons with these methods to demonstrate the superiority of the proposed designs.
[1] Chen et al. Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift.
[2] Ouali et al. Black Box Few-Shot Adaptation for Vision-Language Models.
Questions for authors
Please refer to the Critical Weaknesses.
6 comments
W1 / Q1 — Direct comparisons with MG-MTTA and U-LFA
We evaluated both methods on AnyTouch and UniTouch, keeping their protocols separate. Accuracy (%) averages backbones equally; TAG-C averages its three settings, and Average weights the three dataset families equally.
MG-MTTA: online adaptation. Our vision–touch implementation of MG-MTTA follows the official released code [1], updating encoder normalization parameters and the fusion gate. Both methods receive identical streams at batch size 16 and seed 0; MG-MTTA's released batch size is 64. MG-MTTA adapts before prediction, while SenseAlign updates afterward.
| Method | OFR | RCT Material | TAG-C | Average |
|---|---|---|---|---|
| MG-MTTA | 46.59 | 45.96 | 33.03 | 41.86 |
| SenseAlign | 62.29 | 55.55 | 60.57 | 59.47 |
SenseAlign leads in nine of ten backbone–setting combinations; MG-MTTA leads on UniTouch RCT Material (71.82% vs. 64.95%). The low TAG-C scores include single-class collapse on AnyTouch Visual and Mixed. Freezing the encoders while retaining gate and anchor updates prevents this collapse, implicating encoder drift in the tested configuration. SenseAlign's frozen encoders avoid this failure path.
U-LFA: support fitting, then frozen evaluation. Both methods fit the same unlabeled support set and remain frozen on disjoint query groups. U-LFA [2] fits vision and touch independently, and we average their logits. This gives both methods the same support information and no query-time adaptation.
| Method | OFR | RCT Material | TAG-C | Average |
|---|---|---|---|---|
| U-LFA | 41.35 | 26.26 | 43.66 | 37.09 |
| SenseAlign | 63.52 | 52.25 | 57.79 | 57.85 |
SenseAlign leads in all ten backbone–setting combinations averaged over five seeds. Weak-sensor assignments may distort U-LFA's independently fitted map, whose scores still contribute equally to fusion. Its uniform-class assignment prior may also mismatch support sets that are imbalanced or omit classes. We will clarify the implementations and protocols alongside the results.
References
[1] Chen et al. Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift. ACM MM 2026. Official released configuration.
[2] Ouali et al. Black Box Few-Shot Adaptation for Vision-Language Models. ICCV 2023.
Follow-up on our responses
Dear Reviewer,
Following your suggestions, we have clarified the technical differences from MG-MTTA and U-LFA and added direct experimental comparisons with both methods on AnyTouch and UniTouch. When you have a chance, we would appreciate your feedback on these clarifications and results. We would be happy to discuss any remaining concerns.
Thank you again for your time and consideration.
W1 / Q1 — Technical differences from MG-MTTA and U-LFA
Thank you for recognizing the practical importance of unreliable modalities and the complementary roles of our two components. We are delighted that you highlighted both aspects.
SenseAlign's contribution is a shared semantic construction for online vision–touch adaptation: the same pseudo-labeled history supports both modality reliability estimation and feature correction.
MG-MTTA. Its anchors are running posterior statistics [1]. Drift between sorted current posteriors and these anchors regularizes a learned fusion gate. SenseAlign instead uses fixed class-text anchors: it groups paired observations under shared pseudo-labels, averages each modality's standardized support for its assigned anchor across historical groups, and applies softmax to obtain fusion weights directly. This compares vision and touch against the same class-specific reference when their reliability differs.
U-LFA. It alternates sample-to-class assignment and feature-map fitting [2]. SenseAlign builds shared historical class groups from raw equal fusion: separate visual and tactile statistics estimate reliability, while fused class means fit one constrained correction applied to both sensors. Both operations use prior history, which is updated after each batch's predictions. Procrustes itself is an established mathematical tool; our contribution lies in how the shared semantic history organizes these complementary operations for vision and touch.
We will make these distinctions explicit. Direct comparisons are reported in our response “W1 / Q1 — Direct comparisons with MG-MTTA and U-LFA”.
References
[1] Chen et al. Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift. ACM MM 2026. Official released configuration.
[2] Ouali et al. Black Box Few-Shot Adaptation for Vision-Language Models. ICCV 2023.