- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TacDyn-WAM extends InternVLA-A1 with a tactile dynamics expert and a read-only tactile understanding memory. It learns TacRep using masked spatiotemporal feature prediction and relational distillation, then predicts future tactile features and feature changes at three horizons. Visual, tactile, and action experts interact through joint attention and staged training. The method reports 81.5% mean success across eight UniVTAC simulation tasks, 71.0% across five real-robot tasks, and 85.0% after additional real-data pretraining. Validation includes pathway removal, alternative tactile target spaces, and GPU inference-latency comparisons.
Critical strengths
- The complete system produces substantial improvements in both simulation and real manipulation. It exceeds the reported base model by 25.7 points on UniVTAC and by 31 points in the real-robot evaluation without additional OmniViTac pretraining. Evidence across contact-rich tasks makes the contribution consequential for robotic control rather than only representation prediction.
- The separation of future tactile dynamics from current tactile understanding is coherent and empirically useful. Removing the tactile expert or memory reduces UniVTAC mean success from 81.5% to 67.9% or 71.3%, respectively, and the target-space comparisons favor TacRep over Cosmos VAE and DINOv2. These ablations support the main architectural choices.
Critical weaknesses
- The ablations do not isolate several mechanisms central to the explanation. Table 2 removes whole pathways or swaps target encoders, but does not separately test future versus delta prediction, single versus multiple horizons, relational distillation, or change-based patch weighting. A capacity-matched tactile expert with its prediction losses disabled would also separate auxiliary forecasting from extra tactile processing. Consequently, the results establish the complete system's usefulness more clearly than the claimed causal importance of implicit dynamics learning.
- The evaluation leaves robustness and statistical uncertainty undercharacterized. Real tasks have only 20 trials per policy, with no reported training-seed variability, and initial conditions are sampled within the demonstration workspace (Appendix H.2). UniVTAC uses published baseline results with potentially different execution settings, although the base model and ablations are re-evaluated consistently. These limitations matter for the strength of superiority and deployment-drift claims.
Questions for authors
-
Under matched tactile-expert capacity and training budget, what happens when the future/delta supervision is removed, or when only one horizon is predicted? These comparisons would distinguish dynamics supervision from additional tactile capacity.
-
Are the reported success rates from a single trained checkpoint per task? Please provide any existing variation across training runs and explain whether all real-world methods used comparable action-execution schedules. This affects the interpretation of the success-rate gaps.
2 comments
Response to Reviewer kxm8 (Part 2/2): Robustness, Training Variability, and Execution Protocols
2. Evaluation robustness, training variability, and execution protocols
Expanded real-robot evaluation. We have increased evaluation from 20 to 50 trials per task for all five methods, giving 250 trials per method across five tasks. We also use a broader range of randomized initial object placements, increasing task difficulty and testing generalization under more varied contact conditions. These variations make tactile pixel patterns more likely to depart from the demonstration distribution. Success rates and successful-trial counts are:
| Method | Stack Cups | Remove Plug | Insert Plug | Unscrew Cup Lid | Wipe Whiteboard | Avg. |
|---|---|---|---|---|---|---|
| LingBot-VA | 30% (15/50) | 8% (4/50) | 32% (16/50) | 10% (5/50) | 40% (20/50) | 24.0% |
| InternVLA-A1 | 42% (21/50) | 14% (7/50) | 36% (18/50) | 8% (4/50) | 58% (29/50) | 31.6% |
| FTP-1 | 52% (26/50) | 34% (17/50) | 50% (25/50) | 14% (7/50) | 68% (34/50) | 43.6% |
| TacDyn-WAM | 76% (38/50) | 56% (28/50) | 78% (39/50) | 36% (18/50) | 92% (46/50) | 67.6% |
| TacDyn-WAM (modest pretraining) | 94% (47/50) | 84% (42/50) | 90% (45/50) | 56% (28/50) | 98% (49/50) | 84.4% |
Under these harder conditions, TacDyn-WAM and its modestly pretrained variant decline by only 3.4 and 0.6 percentage points relative to the original evaluation, versus 8.0–11.4 points for the baselines. TacDyn-WAM retains a 24.0-point advantage over FTP-1, the strongest baseline. The larger evaluation therefore supports the performance advantage and robustness to initial-state variation in closed-loop execution.
Training runs and checkpoints. Each reported task result uses one trained checkpoint, evaluated over multiple trials. We do not currently have results from independent training seeds. The expanded evaluation measures execution variability for these checkpoints; it does not estimate variability across training runs. We will make this distinction explicit in the evaluation description.
Sensitivity to the simulation execution horizon. We additionally evaluated executing all 50 predicted steps on Insert Hole and Lift Can, where the original 32-step setting was chosen for contact stability. Success changes only slightly, from 91% to 89% and 87% to 84%, respectively. With the other six tasks unchanged, the eight-task average is 80.9% versus 81.5% originally. This shows that the main performance gains remain with full 50-step execution. The base-model and pathway-ablation comparisons use consistent execution settings; results quoted from external publications retain their original evaluation protocols.
Real-world execution schedules. Both TacDyn-WAM variants, InternVLA-A1, and FTP-1 predict 50-step action chunks and execute the first 32 steps before replanning. LingBot-VA uses its default deployment configuration because its action chunks are coupled to video-latent frames, with observation feedback and KV-cache updates aligned to those chunks. Adopting a 50-step prediction / 32-step execution schedule would require adapting this temporal alignment, rather than simply changing two horizon parameters. We therefore preserve its native deployment protocol.
Response to Reviewer kxm8 (Part 1/2): Predictive Supervision and Component Ablations
Thank you for your careful review and constructive feedback. We address your concerns point by point below.
1. Contributions of predictive supervision, multiple horizons, and RSD
We have added the two ablations highlighted in your question—retaining the tactile expert without its future/delta prediction supervision, and predicting a single horizon—alongside an RSD ablation. These comparisons examine specific mechanisms beyond removing entire pathways or replacing target encoders.
The table reports success rates (%) on seven UniVTAC tasks. Insert HDMI is omitted from this mechanism-focused evaluation because its demonstrations contain nearly no useful tactile signal; it remains included in the original eight-task benchmark results. All averages below use the same seven tasks.
Contribution of predictive supervision. The w/o ITDE predictive training variant retains the expert and its connections for action learning, but removes future/delta prediction supervision. Average success falls by 8.0 points, with the largest decrease on Put Bottle (13 points) and 10-point decreases on Insert Tube, Pull-out Key, and Lift Bottle. Retaining the tactile processing module alone therefore does not recover the full model's performance, supporting the contribution of explicitly training it to predict contact evolution.
Multiple horizons improve control. Predicting only the middle horizon, 15 frames, reduces average success by 6.0 points relative to predicting 5/15/25 frames. The largest decreases occur on Put Bottle (9 points), Insert Tube and Lift Bottle (8 points each), and Pull-out Key (7 points). These tasks require tracking evolving contact during reorientation, alignment, or extraction. The pattern is consistent with the benefit of anticipating contact evolution at multiple temporal scales.
RSD complements temporal prediction. Removing RSD reduces average success by 4.6 points, with the largest drops on Put Bottle (8 points), Insert Tube and Lift Bottle (6 points each). As discussed in the paper, these tasks involve sustained rotation of the contact pattern. RSD preserves local spatial relations while TDP learns temporal evolution, helping the representation capture contact dynamics as tactile pixel patterns change. The ablation supports the contribution of this spatial constraint beyond temporal predictive learning alone.