acceptodds
Withdrawn from ICLR 2027

DATE: Dual-scale Adaptive Temporal Encoding for Tactile Property Perception

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-based tactile sensing enables robots to infer physical properties from contact interactions. Yet the cues needed for different properties unfold at different temporal scales. Hardness is reflected in deformation accumulated during pressing, whereas roughness is revealed by local texture and shear changes during sliding. Models that rely on a single global summary or a fixed short window can miss complementary evidence needed for these properties. To address this, we propose Dual-scale Adaptive Temporal Encoding DATE, which jointly models local contact changes and full-sequence dynamics. A frozen tactile backbone provides patch and frame-level features. A local branch applies spatiotemporal attention over patch tokens within short windows to preserve texture and shear transitions, while a global branch encodes the deformation history across the observed sequence. Property-specific gated fusion combines these temporal representations with statistical cues, allowing hardness and roughness to draw on different temporal evidence across pressing, sliding, and twisting. We further integrate the learned tactile representation with a frozen vision-language model for material description and property reasoning. Evaluations on two benchmarks with graded property labels demonstrate the effectiveness of DATE. On VitaSet, it improves hardness and roughness classification accuracy over AnyTouch by 7.38 and 5.14 percentage points, respectively. The downstream extension achieves an overall TVL description score of 5.66, exceeding TVL-LLaMA (ViT-Base) by 0.63 points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision–tactile fusion, Temporal feature modeling, Material property recognition, Hardness and roughness classification, Multimodal perception

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cBWAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes Dual-scale Adaptive Temporal Encoding (DATE), which jointly models local contact changes and full-sequence dynamics. Specifically, the work focuses on two physical properties with distinct temporal characteristics: hardness, which relies on long-term deformation accumulation during pressing, and roughness, which is reflected by short-term local texture and shear variations during sliding. Building upon a frozen AnyTouch backbone, this work introduces additional temporal modeling modules to capture property-specific temporal evidence, enabling more accurate tactile property classification.

Critical strengths
  1. The motivation is clear and physically intuitive.
  2. The method can handle variable-length tactile sequences and maintain cached representations.
Critical weaknesses

The two most significant weaknesses of this work are: (1) inadequate coverage and discussion of related work, and (2) flaws in the experimental evaluation.

Specifically:

  1. Insufficient discussion of related video understanding works. The paper lacks a comprehensive discussion of prior video understanding approaches. To my knowledge, jointly modeling global and local spatio-temporal features has already been widely explored in video understanding, e.g., ViViT [1]. Although existing methods differ in their specific designs, the authors do not explain why these video understanding architectures cannot be directly applied to tactile attribute understanding, nor what unique challenges arise in the tactile domain. More importantly, the most relevant work to this paper appears to be GLFormer [2], which was submitted 1 year ago and published 6 months ago, and therefore should not be considered a concurrent work. The proposed architecture is highly similar to GLFormer, with the main difference seemingly being a straightforward domain transfer from visual videos to tactile sequences. The paper does not clearly identify the fundamental technical differences or novel insights beyond this adaptation. In addition, the recent tactile temporal modeling approach VTV [3], which specifically targets tactile attribute understanding, is not discussed at all. Ignoring these closely related works raises significant concerns regarding the novelty and technical soundness of this paper. The claim that “No existing model preserves short patch-level events and full interaction evolution in parallel, or adapts their relative contributions to each property” appears to be overstated. This statement seems to be based only on the fact that such approaches have not been directly adopted in the tactile domain, rather than on a fundamental limitation of existing methods.

  2. Serious concerns regarding the cross-dataset evaluation in Table 2. The results reported in Table 2 are highly questionable. First, the performance of the VitaSet → PhysiCLeAR transfer setting is unexpectedly low. For a three-class classification task, a random classifier should achieve approximately 33% accuracy. However, none of the methods in the Hardness column reaches this level, and similar issues can be observed in other reported results. This raises concerns about whether the evaluation protocol, label definitions, or metric calculation are correctly implemented. Second, some results in the PhysiCLeAR → VitaSet setting appear to be multiples of 1/37, and their standard deviations are exactly 0. Does this indicate that the test set contains only 37 samples? If so, the evaluation may not be statistically reliable, and the reported performance could be highly sensitive to individual samples. These issues suggest serious problems with the cross-dataset evaluation protocol. The authors should provide detailed information about the test set size, class distribution, evaluation procedure, and metric computation, and clarify why the reported results are reliable.

  3. Insufficient experimental details. The paper does not provide sufficient details on how different baseline encoders are trained and evaluated on the downstream tasks. For example, it is unclear how the downstream task heads are designed for different baseline encoders. In particular, the statement “TVL uses its original evaluation protocol” is unclear. Does this indicate that TVL adopts a different evaluation setting from other methods? In addition, the data splits used for downstream evaluation are not clearly specified. The authors need to provide a detailed explanation and ideally report results under a unified evaluation setting.

  4. Insufficient comprehensiveness of comparisons and ablation studies. To validate the superiority of the proposed temporal encoding module, the authors should include comparisons with simpler temporal modeling approaches, such as applying an LSTM or other standard temporal aggregation modules on top of AnyTouch features. The ablation studies are also not sufficiently comprehensive. Several key components of the proposed architecture, including BiGRU, 3D RoPE, and the gating mechanism, should be individually ablated to quantify their respective contributions. Moreover, the paper mentioned AnyTouch 2 but does not report its performance.

  5. Lack of analysis on the gating mechanism. The paper lacks a direct analysis and discussion of the gate outputs. For example, the authors should provide visualizations of the gate outputs in different tactile attribute recognition tasks. Such analysis would serve as direct evidence supporting the motivation that different attributes rely on different types of features. A few Grad-CAM visualizations is far from sufficient to validate the proposed motivation.

  6. Unfair comparison of inference time and model size in Table 5. The comparison of inference time and model size in Table 5 appears to be unfair. The authors state that “We evaluate DATE’s runtime using small pre-trained encoder and post-encoder modules at test time,” which seems inconsistent with the actual model used in the main experiments, where DATE is built upon the AnyTouch encoder.

  7. Unfair comparison on the TVL benchmark. The comparison in Table 6 is not fair because the proposed method adopts a much more advanced and powerful LLM backbone, Qwen3-VL-8B-Instruct, while previous methods are built upon earlier-generation LLMs (e.g., LLaMA). The performance improvement may therefore come from the stronger language model backbone rather than the proposed method itself. To enable a fair comparison, the authors should either adopt the same LLM backbone as previous methods or re-implement existing methods with the same Qwen3-VL-8B-Instruct backbone.

  8. No experimental results for robot interaction and object ranking in Appendix D.

  9. Figure 4 is visually cluttered and difficult to interpret. The current design makes it challenging for readers to quickly identify the key experimental findings and compare the performance differences among different methods.

Overall, there are serious issues with the related work discussions and experimental evaluations. The main conclusions of this paper are not sufficiently supported by the current presentation and experiments.

[1] Arnab, Anurag, et al. "Vivit: A video vision transformer." 2021 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2021.

[2] Wu, Hanbo, et al. "GLFormer: a hierarchical cross-layer transformer decoding framework with global–local feature collaboration for video action recognition." Complex & Intelligent Systems 12.7 (2026): 171.

[3] Xie, Yifan, et al. "Universal visuo-tactile video understanding for embodied interaction." Advances in Neural Information Processing Systems 38 (2026): 127864-127883.

Questions for authors

For tactile attributes beyond hardness and roughness, such as the important contact force, which category should they belong to?

Reviewer_JKaJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DATE for predicting graded hardness and roughness from vision-based tactile sequences. A frozen AnyTouch backbone supplies patch and frame features. Short-window spatiotemporal attention models local changes, a BiGRU models the observed frame history, and property-specific gates combine these features with a statistical anchor. The encoder is evaluated on VitaSet and PhysiCLeAR variants, including action-wise, label-granularity, and cross-dataset settings. On VitaSet, the reported hardness/roughness accuracies are 80.76%/75.39%, compared with 73.38%/70.25% for AnyTouch. A Qwen3-VL-based downstream extension obtains a TVL description score of 5.66; property-ranking demonstrations are also provided.

Critical strengths
  • The design has a clear task-based motivation, with supporting ablation evidence. Hardness and roughness plausibly benefit from different temporal cues. In Table 3, removing the global branch reduces hardness accuracy from 80.76% to 75.84%, whereas removing the local branch reduces roughness accuracy from 75.39% to 72.48%. These results support the practical value of complementary branches.
  • The reported classification improvements are meaningful and evaluated beyond one aggregate accuracy. Table 1 reports gains of 7.38 and 5.14 percentage points over AnyTouch, with improvements in Macro-F1 and three-seed reporting. Action-wise evaluations and three-/four-level classification extend the evidence beyond the primary VitaSet comparison. The frozen-backbone design also makes the proposed addition practically appealing.
Critical weaknesses
  • The evaluation protocol is not specified well enough to assess the central generalization and temporal-modeling claims. Section 4.1 describes VitaSet as 21,510 tactile images but does not explain how these become chronological interaction sequences, including sequence boundaries, sampling, window stride, and treatment of short sequences. It states that methods share a split without giving split sizes or whether objects and contact interactions are disjoint. The baseline input frames, temporal aggregation, and training/freeze policies are also unclear. These details matter because repeated views of the same object or unequal temporal information could substantially affect the reported gains.
  • The experiments only partially isolate the claimed adaptive temporal contribution. Table 3 is encouraging, but the internal no-temporal variant already achieves 75.62%/72.93%; the full model adds 5.14/2.46 points to this control, rather than the headline 7.38/5.14 points over standalone AnyTouch. There is no fixed-fusion control establishing the benefit of property-specific adaptive gating, or an order-control experiment distinguishing temporal structure from additional learned transformations. For downstream descriptions, Table 6 also changes the language/vision backbone to Qwen3-VL, without a comparison using the same Qwen model and a static tactile encoder. Thus, the 0.63-point gain cannot be attributed specifically to DATE. These limitations leave the added scientific value of the proposed design insufficiently established.
Questions for authors
  • How are VitaSet images assembled into chronological interaction sequences, and what are the sampling policy and sequence lengths? Please provide train/validation/test sizes and object/interaction identifiers, clarify whether objects are shared across splits, and give per-class counts by data source, particularly for the Medium-hardness annotations. What frame inputs, aggregation, and training policies were used for the baselines?
  • Table 3 already provides moment-pooling and global-branch controls. Within this setting, how much do the property-specific learned gates improve over fixed fusion? Does an order-shuffled control using the same frames and training setup preserve the gains? Reporting means and variability for these targeted controls would materially clarify the specific adaptive temporal contribution.
  • How is temporal context obtained for the 402 TVL image-touch test pairs, and how is it kept separate from training data? Please specify the tactile connector's training data/objective and the source of semantic labels in the prompt. Under the same Qwen3-VL model, visual inputs, and prompt policy, does DATE outperform a static/no-temporal tactile representation? This would clarify whether Table 6 validates the temporal encoder itself.
Reviewer_UvmtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose Dual-scale Adaptive Temporal Encoding (DATE), jointly modelling local contact changes and full-sequence dynamics. The local branch uses spatiotemporal attention over patch tokens wthin short window, while the glocal branch uses a bidirectional recurrent model to process the full sequence. A a temporal-dynamics-aware gated fusion method fuses local, global and statistical features. DATE is evaluated on TVL and VitaSet, performing better than sota baselines.

Critical strengths
  1. The paper addresses a meaningful limitation of tactile sequence representations: hardness and roughness rely on different temporal cues. The distinction between local texture changes and long-term deformation gives a reasonable motivation for the architecture.

  2. DATE outperforms all compared methods on VitaSet, improving over AnyTouch by 7.38 and 5.14 percentage points in hardness and roughness accuracy.

  3. The experiments cover multiple interaction types, cross-dataset transfer, label granularity, and a downstream vision-tactile-language task.

Critical weaknesses
  1. The architecture combines established components: spatiotemporal attention, a BiGRU, statistical pooling, and MLP gating. There is little technical contribution beyond their integration.

  2. There is no comparison against concatenation, fixed-weight fusion, or a shared gate. The ablated variants in Table 3 still use the gate, and the learned gate weights are never reported. As a result, the claimed property-dependent preferences are unverified.

  3. The temporal contribution is unclear. The anchor-only variant (75.62% / 72.93%) already beats AnyTouch. Much of the headline gain, including over half of the roughness gain, comes from the anchor and head rather than temporal modeling.

  4. The downstream gains are not isolated. Baselines use different language models than Qwen3-VL-8B, and there is no same-LLM control with a non-temporal encoder.

Questions for authors
  1. Can the authors report results with temporally shuffled frames, for both the full model and each branch?

  2. Can the authors compare the gated fusion against concatenation, fixed-weight averaging, and a shared gate, and report the distribution of learned α and γ values per property and per action type?

  3. Which backbone is used for the runtime in Table 5, and are the accuracy results obtained with the same configuration?

  4. Why does Mixed → Mixed (88.54% / 91.08%) substantially exceed within-VitaSet performance (80.76% / 75.39%)? How were the mixed splits constructed, and could objects overlap between train and test?

  5. What is the performance of the Qwen3-VL pipeline with a plain AnyTouch encoder in place of DATE?

  6. Can the authors provide quantitative results for the three-object ranking task?

  7. Why do several cross-dataset entries show zero variance across seeds?

  8. How many parameters are from anytouch ecnoder? Why is DATA so much faster than antouch during inference if they are the same encoder?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.