- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces TouchBench, a benchmark containing 10K questions across six tactile task categories, and TouchAgent, a framework combining tactile perception tools, knowledge retrieval, and numerical reasoning. The authors additionally construct 40K interaction trajectories and train the agent through supervised fine-tuning and evidence-grounded reinforcement learning. The results show substantial improvements over the evaluated general-purpose and tactile-language models.
Critical strengths
- TouchBench represents a substantial curation effort, integrating observations from multiple datasets and sensor types with expert annotation and verification.
- The integration of tactile perception, external knowledge, and computation is sensible. The training-stage and tool-component ablations provide useful evidence for the proposed system.
Critical weaknesses
- A particularly relevant omission is Octopi-1.5, which supports tactile observations from multiple object parts and incorporates retrieval-augmented generation. The paper only compares against the original Octopi, leaving its advantage over an existing retrieval-enhanced tactile-language system unclear. Although Figure 2 lists TactileBench, the associated SToLa framework, which addresses tactile interaction perception and open-ended commonsense reasoning, is not substantively discussed. The distinction between these reasoning capabilities and those evaluated by TouchBench deserves a clearer explanation. Furthermore, Touch-R1 is cited but not evaluated, despite its closely related emphasis on tactile-grounded reinforcement learning. These omissions weaken the assessment of the paper’s contribution relative to existing tactile reasoning approaches. Empirical comparisons should be included where implementations and input compatibility permit, with explicit explanations for exclusions.
- Several evaluated models, including InternVL2.5-8B, LLaVA-OneVision-Qwen2-7B, and MiniCPM-V-2.6, represent older generations, while the open-source and tactile-language comparisons are concentrated around 7–8B parameters. Although Qwen3-VL-8B and several strong proprietary models are included, the paper does not examine scaling within an open model family. Larger alternatives, such as Qwen3-VL-32B or Qwen3-VL-235B-A22B, and newer InternVL generations such as InternVL3.5, would provide a more informative evaluation. TouchAgent itself is tested only with a Qwen2.5-7B controller, leaving its dependence on controller capacity and generation unclear.
- The main results compare a specialized tool-augmented system against models evaluated through direct multiple-choice prompting. TouchAgent additionally benefits from domain-specific perception models, curated knowledge, and trajectory training. The base TouchAgent and component ablations are useful, but they do not establish an advantage over a strong general-purpose agent given the same resources. This is particularly relevant because KnowledgeSearch can itself invoke GPT-5.2 during inference, as described in Appendix A.3.3. A comparison with a GPT-5.2- or Claude-based agent using the same tools, knowledge base, and interaction budget would better establish the contribution of the proposed planning and training approach.
Questions for authors
- How does TouchBench differ from SToLa’s TactileBench in the reasoning capabilities it evaluates, beyond scale and tool-trajectory annotations? Could you include comparisons with Octopi-1.5 using retrieval and with Touch-R1 on compatible tasks, or explain the constraints preventing these comparisons?
- Could you extend the evaluation to larger open-source models and newer model generations, and test TouchAgent with a newer or larger controller while keeping the tool environment fixed? Do the reported capability gaps and training gains persist as model capacity increases?
- How does TouchAgent compare with a strong general-purpose agent using the identical toolkit, knowledge base, and tool-call budget? Please also clarify the frequency and computational cost of GPT-5.2 calls within KnowledgeSearch.
7 comments
Response to Reviewer GRDS
W2 & Q2: Model Scale and Controller Scalability.
We agree with this suggestion and have added experiments on both expanded baselines and alternative controllers to further validate the effectiveness and scalability of our method.
(1) Comparisons with newer and larger baseline models. Following the reviewer's suggestion, we have included newer general-purpose MLLMs, namely
InternVL3.5-8B,LLaVA-OneVision-2-8B-Instruct, andMiniCPM-V-4.5 (8B), as well as larger tactile-language models, namelyOctopi-13B,VTV-LLM-14B, andTouchThinker-14B, under the same TouchBench evaluation protocol. In addition, we have includedQwen3-VL-32BandQwen3-VL-235B-A22Balongside the previously evaluated Qwen3-VL-8B to compare different scales within the same model family. As shown in Table 2, TouchAgent with a 7B controller remains highly competitive against newer general-purpose MLLMs and larger tactile-language models, demonstrating the effectiveness of our method.
Table 2: TouchBench evaluation results with expanded baselines.
| Model | Attr. | Match. | Inter. | Scene. | Dyn. | Cross. | Avg. |
|---|---|---|---|---|---|---|---|
| Open-source MLLMs | |||||||
| InternVL3.5-8B | 16.2 | 31.1 | 20.7 | 36.2 | 26.5 | 31.7 | 27.1 |
| LLaVA-OneVision-2-8B-Instruct | 22.9 | 28.3 | 24.2 | 32.7 | 34.8 | 32.8 | 29.3 |
| MiniCPM-V-4.5 (8B) | 16.2 | 27.9 | 20.7 | 36.2 | 36.8 | 22.7 | 26.8 |
| Qwen3-VL-8B | 24.3 | 34.8 | 23.7 | 28.5 | 34.1 | 31.5 | 29.5 |
| Qwen3-VL-32B | 26.1 | 32.4 | 24.5 | 27.8 | 36.9 | 34.7 | 30.4 |
| Qwen3-VL-235B-A22B | 26.4 | 33.7 | 29.3 | 37.5 | 39.7 | 38.4 | 34.2 |
| Tactile-Language Models | |||||||
| Octopi-13B | 57.6 | 52.1 | 38.6 | 45.4 | 39.5 | 39.1 | 45.4 |
| VTV-LLM-14B | 61.2 | 63.8 | 41.4 | 47.3 | 43.8 | 40.1 | 49.6 |
| TouchThinker-14B | 63.8 | 60.5 | 49.2 | 54.6 | 50.9 | 42.8 | 53.6 |
| TouchAgent-7B | 78.6 | 74.6 | 77.9 | 73.8 | 76.3 | 75.4 | 76.1 |
(2) Comparisons across controller scales. To assess the dependence of our method on controller capacity and model generation, we extended the evaluation to six controller configurations:
Qwen2.5 (7B, 14B, and 32B)andQwen3 (8B, 14B, and 32B), each evaluated at the Base, SFT, and SFT+RL stages. All settings use the same tool environment, knowledge base, training data, and evaluation protocol. The Base models retain access to the tools and knowledge base but receive no training. As shown in Table 3, SFT improves macro-average accuracy by at least 5 percentage points (pp) across all six controllers, while RL provides a further gain of at least 3 pp over SFT. The average gains from the two stages are 5.7 pp and 4.1 pp, respectively. These results demonstrate that TouchAgent's training benefits extend across controller scales and model generations.
Table 3: Comparison of training stages across controllers of different capacities and model generations. All values are macro-averaged accuracies (%) across the six TouchBench task categories.
| Controller | Base | SFT | ΔSFT | SFT+RL | ΔRL |
|---|---|---|---|---|---|
| Qwen2.5-7B | 66.2 | 71.9 | +5.7 | 76.1 | +4.2 |
| Qwen2.5-14B | 65.7 | 72.1 | +6.4 | 75.5 | +3.4 |
| Qwen2.5-32B | 68.5 | 73.6 | +5.1 | 78.3 | +4.7 |
| Qwen3-8B | 67.1 | 73.3 | +6.2 | 77.2 | +3.9 |
| Qwen3-14B | 67.9 | 73.1 | +5.2 | 76.9 | +3.8 |
| Qwen3-32B | 69.2 | 74.8 | +5.6 | 79.4 | +4.6 |
We have added the above experimental results to Table 1 in the main text and Appendix A.7 of the revised manuscript (highlighted in red).
Response to Reviewer GRDS
W3 & Q3: Comparisons with Strong General-Purpose Agents under the Same Resources and Additional API Costs
We thank the reviewer for these helpful suggestions and have conducted additional experiments and provided clarifications to address them.
(1) Comparisons with strong general-purpose agents under the same resources. We replaced the original Qwen2.5-7B with
GPT-5.2andClaude-Opus-4.6as Agent controllers, while keeping the toolset, knowledge base, and interaction budget identical. Neither general-purpose model receives additional training. As shown in Table 4, both general-purpose agents outperform the untrained Qwen2.5-7B Base (66.2%), indicating that controller capability contributes to tool-augmented reasoning. However, trajectory-supervised SFT raises TouchAgent-7B's accuracy to 71.9%, and subsequent RL further improves it to 76.1%, exceeding the strongest general-purpose agent, Claude Opus 4.6 (69.9%), by 2.0 pp and 6.2 pp, respectively. These results demonstrate that specialized interaction-trajectory training and reinforcement learning provide additional benefits for tool planning and evidence integration under identical domain-specific resources. We have added the above experimental results to Table 8 in Appendix A.7.1 of the revised manuscript (highlighted inred).
Table 4: Comparison of agents using the same toolset, knowledge base, and interaction budget.
| Model | Attr. | Match. | Inter. | Scene. | Dyn. | Cross. | Avg. | Δ (vs. TouchAgent (SFT+RL)) |
|---|---|---|---|---|---|---|---|---|
| GPT-5.2 | 71.3 | 67.6 | 69.1 | 65.4 | 68.9 | 67.3 | 68.3 | +7.8 |
| Claude Opus 4.6 | 72.8 | 68.1 | 70.2 | 69.2 | 70.3 | 68.5 | 69.9 | +6.2 |
| TouchAgent (Base) | 69.2 | 64.7 | 67.4 | 64.2 | 66.1 | 65.5 | 66.2 | +9.9 |
| TouchAgent (SFT) | 74.8 | 71.5 | 73.1 | 70.4 | 71.6 | 70.1 | 71.9 | +4.2 |
| TouchAgent (SFT+RL, Ours) | 78.6 | 74.6 | 77.9 | 73.8 | 76.3 | 75.4 | 76.1 | — |
(2) Frequency and cost of GPT-5.2 calls within KnowledgeSearch. As described in Appendix A.4.3, KnowledgeSearch prioritizes exact matching and semantic retrieval to obtain knowledge, calling GPT-5.2 only when the retrieved results are incomplete or additional physical explanations are needed. We recorded the calls made by TouchAgent-7B on the full 10K test set: 1,349 questions triggered GPT-5.2 calls within
KnowledgeSearch, accounting for 13.49% of the test set. These calls incurred an additional computational cost of approximately USD 34.2.
Response to Reviewer GRDS
We sincerely thank the reviewer for the constructive feedback and positive assessment of our paper’s presentation, informativeness, and soundness. We have carefully considered all comments and revised the manuscript accordingly.
W1 & Q1: Benchmark Differences and Comparisons with Related Methods.
We sincerely thank the reviewer for the constructive comments. We further clarify the benchmark differences and add comparisons with related methods.
We have further clarified the distinctions from related work in Section 2 and added the corresponding experimental results in Table 1 of the revised manuscript (highlighted in
red).Table 1: Comparison with related tactile reasoning methods.
Reference
[1] STOLA: Self-Adaptive Touch-Language Framework for Tactile Commonsense Reasoning in Open-Ended Scenarios. AAAI 2026.
[2] Octopi: Object property reasoning with large tactile-language models. RSS 2024.
[3] Universal visuo-tactile video understanding for embodied interaction. NeurIPS 2025.
[4] Touchthinker: Scaling tactile commonsense reasoning to the open world with large-scale data and action-aware representation. EMNLP 2026.
[5] Touch-R1: Reinforcing Touch Reasoning in MLLMs. NeurIPS 2026.
[6] Demonstrating the Octopi-1.5 Visual-Tactile-Language Model. RSS 2025.