acceptodds
Under review as a conference paper at ICLR 2027

TAD: Teacher-Assistant Distillation of Tactile Predictions for Robot Control

Abstract

Tactile sensing provides complementary information for contact-rich manipulation, and predictions of tactile change offer an additional source of supervision for robot policies. However, forecasting and control impose different requirements on a representation, making it challenging to preserve predictive knowledge while adapting it to action learning. We propose Teacher-Assistant Distillation (TAD), which transfers tactile predictions into a vision–language–action policy through a frozen teacher and an adaptive assistant. The teacher predicts features of future tactile changes from current observations and initially supervises intermediate predictive tokens in the policy. During training, a teacher-initialized assistant adapts through action supervision while remaining anchored to the teacher. An exponential moving average of the assistant's prediction parameters provides distillation targets, while the online assistant's action readout supplies residual corrections to the denoising field. Asymmetric attention isolates predictive tokens from noisy-action inputs while allowing action tokens to attend to them. Instantiated on , TAD achieves mean success rates of 90.0% across eight UniVTAC tasks and 87.5% across six real-world tasks, exceeding -VTLA by 6.9 and 10.0 percentage points, respectively. Ablations show lower success rates without the assistant, teacher-derived supervision, or future tactile targets. These findings support teacher-anchored adaptation as a mechanism for transferring tactile predictive knowledge into robot control.

Primary area:
generative models
Keywords:
Tactile Prediction, Vision-Language-Action Models, Predictive Representation Learning, Contact-Rich Manipulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_efYxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TAD introduces a mechanism for adapting predictive tactile knowledge to robot control. A teacher first learns to predict DINOv3 features of future tactile changes from current observations, then supervises intermediate predictive tokens in a π0.5-based policy. A teacher-initialized assistant subsequently adapts through action supervision while remaining anchored to the frozen teacher. Its EMA supplies evolving distillation targets, while its online features provide residual corrections to the action-denoising field. Asymmetric attention lets action tokens read predictive tokens while preventing reverse information leakage. The core contribution is the combination of preserved predictive supervision and action-driven adaptation. The deployed policy retains the assistant, achieving reported average success rates of 90.0% in simulation and 87.5% on physical tasks.

Critical strengths

The teacher-anchored assistant addresses a meaningful tension between preserving predictive representations and adapting them for control. The asymmetric attention design carefully prevents action-input leakage.

Evaluation spans eight simulation and six physical tasks, with consistent improvements and substantive teacher, assistant, and future-target ablations.

Critical weaknesses

Removing the assistant simultaneously removes adaptive targets, auxiliary supervision, and residual corrections. The experiments therefore cannot establish whether adaptive distillation itself drives the improvement.

I am confused about the archi, especially about the main figure. I strongly recommend the authors to draw a more clear overview, it cannot highlight your special designs with so many redundant information.

Deployment retains the assistant and its frozen visual-language context, but latency and memory costs are unreported. Main success-rate comparisons also lack uncertainty across independently trained models.

Questions for authors
  1. Can you independently ablate EMA targets, teacher anchoring, and assistant residual corrections while retaining comparable model capacity and training budgets? Which component explains the gain over direct frozen-teacher distillation?

  2. Can prediction and action-correction diagnostics be repeated on episodes withheld from every training stage? The current teacher holdout inherits representation exposure, while correction diagnostics use training demonstrations.

  3. What are the end-to-end inference latency, peak memory, and total training cost relative to N0-VTLA, including the deployed assistant and its frozen visual-language context?

Again, I strongly recommend the authors to draw a more clear overview, if the authors could give a clearer model illustration, I will consider raising my score.

4 comments

Clarifying the architecture in Figure 2

Thank you for pointing out that the original figure made the architecture difficult to follow. We redrew Figure 2 and omitted lower-level implementation details to highlight how the policy uses current touch and learns to anticipate changes in contact.

  1. Learn to predict touch (top). The upper-left panel encodes the change between the initial and current fingertip images. From current observations, the teacher predicts features of the next tactile change. Recorded future images provide the target only during teacher pretraining. In Stage I, we freeze the teacher and train the π₀.₅ student’s predictive tokens to match its forecasts.
  2. Adapt the prediction for control (center and right). In Stage II, the assistant starts from the teacher’s predictor and learns from demonstrated actions. The frozen teacher anchors the assistant’s forecasts, while an exponential moving average (EMA) of the assistant’s prediction parameters supplies updated targets for the student. The asymmetric mask lets action tokens read the student’s predictions while blocking noisy action inputs from reaching them.
  3. Generate the action (bottom right). The student uses its predictive tokens and reads current tactile features through Spatial Readout to compute the main action field. The online assistant’s Action Readout turns its forecast into a residual correction. The yellow plus sign shows where the two are added at each denoising step to produce the action chunk.

Q1. Which components explain the gain over direct frozen-teacher distillation?

We changed one Stage II mechanism at a time, retaining the assistant architecture, auxiliary action supervision, and policy-update budget in all three ablations.

  1. EMA target. We replaced the student’s target from the exponential moving average (EMA) of the assistant with the frozen teacher’s prediction. The online assistant, its anchor, and its action residual remained active.
  2. Teacher anchoring. We retained the assistant’s initialization from the teacher and set only the anchoring loss weight to zero.
  3. Assistant action residual. We set the assistant’s residual correction to zero during both Stage II training and evaluation. The student’s Spatial Readout remained active.

Table E1. UniVTAC success rates (%), with 100 evaluation episodes per task and policy.

TaskDirect teacher distillation, no assistantFixed teacher target, assistant retainedFull TADNo teacher anchorNo assistant residual
Lift Bottle999910098100
Pull-out Key9095997696
Lift Can8091958696
Put Bottle8385908492
Insert Hole9599988487
Insert HDMI3235442534
Insert Tube9998959496
Grasp Classify1001009910099
Mean84.7587.7590.0080.8887.50

Where the gain comes from. Full TAD exceeded direct frozen-teacher distillation by 5.25 percentage points (84.75% to 90.00%). With the teacher target fixed, retaining the assistant and its action branch raised success to 87.75% (+3.00 points). Switching the student to the adaptive EMA target then raised success to 90.00% (+2.25 points). Among the single-component ablations, removing the teacher anchor produced the largest drop (90.00% to 80.88%). Removing the residual reduced success to 87.50%, including a drop from 44% to 34% on Insert HDMI. These last two comparisons measure each component’s effect with the others active, so their drops cannot be added to partition the overall gain.

Q2. Do the tactile-prediction and action-correction results hold on episodes withheld from every training stage?

Yes. We repeated both diagnostics on complete episodes excluded from every training stage, including tactile representation adaptation and policy initialization.

1. Future tactile prediction. Given current observations, we scored the frozen teacher and the trained online assistant against features of the observed tactile change from the current frame to the frame ten steps later. We report cosine distance averaged equally across the eight UniVTAC tasks (lower is better).

Frames evaluatedValid framesFrozen teacherOnline assistantTraining-target mean baseline
All valid frames13,6220.06690.07520.1731
Subset with measurable tactile change12,7420.06460.09330.1465

The teacher had the lowest error on both frame sets. The assistant also outperformed the baseline that always predicts the mean training target, including on frames with measurable change. This supports retention of future-touch information after action training.

2. Assistant action correction. At each checkpoint, we evaluated ten held-out Insert HDMI demonstrations. Within each pair, we fixed the observations, noisy actions, flow times, and target velocities, and toggled only the assistant’s residual correction.

Checkpoint (updates)Residual off (mean MSE)Residual on (mean MSE)Relative reduction
2,0000.015740.0085645.6%
5,0000.012510.0036570.8%

Enabling the residual reduced offline action-field error, measured by normalized flow-matching MSE, on unseen HDMI demonstrations at both checkpoints.

Q3. What are the end-to-end inference latency, peak GPU memory, and total training cost of TAD?

For the fully deployed TAD policy, end-to-end inference took 623.2 ms per action chunk, and peak GPU memory usage was 16.66 GB. Both measurements include the online assistant and its frozen vision–language and tactile context. Training across all five stages used 72.74 H100 GPU-hours in total.

Reviewer_JFcAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TAD adds tactile-prediction tokens to a touch-conditioned policy and supervises them first with a frozen teacher that forecasts future tactile changes, then with an EMA of a teacher-initialized assistant that adapts through the action loss and also corrects the actions directly. It is evaluated on simulation and real-world tasks, with ablations on the real robot.

Critical strengths
  • Keeping a frozen predictive teacher while letting an anchored assistant adapt to the action objective is an interesting idea for transferring tactile predictions into a policy.
  • The real-world ablations support the main components, and replacing future with current tactile targets clearly hurts performance.
Critical weaknesses
  • The paper reads contrived and I couldn't follow all the details very well. The method stacks many components, while the ablations only remove whole groups at once, so it's hard to tell which parts are actually needed.
  • The experiments are quite simple, and the evaluation protocol isn't detailed enough to judge the comparisons. In simulation, the strongest baseline is already close to saturation on most tasks, so the gains come mainly from two of them.
Questions for authors
  1. How many evaluation episodes per task were used in simulation, and how many demonstrations were collected for each real-world task?
  2. Could you report confidence intervals in the evaluation results?
  3. The paper reports prediction error only for the teacher. Could you report the same error for the trained assistant, to show it still predicts future touch rather than collapsing to a generic action module?
3 comments

Weakness 1 — The architecture is difficult to follow, and grouped ablations make individual component contributions unclear

Architecture. TAD teaches a π₀.₅ policy to use both current touch and a forecast of how contact will change. In Stage I, the student learns to match a frozen teacher’s forecasts. In Stage II, an assistant initialized from the teacher learns from demonstrated actions, while the frozen teacher anchors its forecasts. A moving-average copy of the assistant (EMA) supplies the student’s new targets. At deployment, Spatial Readout gives the student direct access to current tactile features, and the online assistant uses its forecast to correct each action-denoising step.

Component ablations. We agree that removing the entire assistant combines several changes. We therefore added the following ablations on eight UniVTAC tasks, changing one design choice at a time. Each policy was evaluated for 100 episodes per task. Full TAD achieved 90.00% mean success.

Design choiceChange relative to full TADMean success (%)
Future-touch supervisionKeep current touch as input and use current tactile features as the training target78.38
Asymmetric attentionAllow predictive tokens to read noisy action tokens78.63
Spatial ReadoutRemove the student’s direct read from spatial tactile memory87.00
Adaptive distillation targetReplace the EMA target with the frozen teacher’s forecast, keeping the online assistant87.75
Teacher anchoringRemove only the anchoring loss, retaining teacher initialization80.88
Assistant action residualSet the assistant’s correction to zero during Stage II training and evaluation87.50

The largest drops came from replacing future targets with current targets and letting predictions access noisy actions. With the assistant retained, adapting the student’s target contributed 2.25 percentage points (90.00% versus 87.75%). Removing the entire assistant and distilling directly from the frozen teacher produced a larger 5.25-point drop (90.00% to 84.75%). These comparisons distinguish the contribution of adaptive supervision from that of the assistant as a whole.

Weakness 2 — Evaluation protocol and concentration of simulation gains

Evaluation protocol. We used all eight UniVTAC tasks [1], as did N₀-VTLA [2]. FTP-1 [3] evaluated six of them. On the full eight-task suite, VTAM [4] reported 83.4% mean success with a visuo-tactile world-action model, and RATE [5] reported 71.8% with risk-aware tactile encoding. For each policy we evaluated, we ran 100 closed-loop episodes per task using the benchmark’s success criteria and weighted all tasks equally. On the real robot, we collected 100 training demonstrations per task and ran 20 separate evaluation trials per policy on each of six tasks.

Simulation results. We agree that most of the gain came from the two tasks with the lowest success rates for N₀-VTLA [2]. TAD improved success on Put Bottle from 60% to 90% and on Insert HDMI from 25% to 44%, while maintaining 95–100% success on each of the other six tasks. Across all eight tasks, mean success rose from 83.1% to 90.0%.

References

[1] UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarking. arXiv:2602.10093v1.

[2] N₀-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens. arXiv:2607.23782v1.

[3] FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation. arXiv:2606.13102v4.

[4] VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining. arXiv:2603.23481v2.

[5] RATE: Risk-Aware Tactile Encoding for Contact-rich Robotic Manipulation. arXiv:2610.03538v1.

Q1. How many evaluation episodes were run per simulation task, and how many demonstrations were collected per real-world task?

In simulation, we ran 100 evaluation episodes per task for each policy we evaluated, across eight tasks. For each of the six real-world tasks, we collected 100 independent demonstration trajectories for training and conducted 20 separate evaluation trials per policy.

Q2. Could you report 95% confidence intervals for the evaluation results?

For each task, we computed a two-sided 95% Wilson interval from the number of successes in 100 simulation episodes or 20 real-robot trials. We then combined the task-specific intervals using MOVER with equal task weights to obtain a 95% confidence interval for full TAD’s mean success rate at the evaluated checkpoints.

Simulation taskSuccess rate [95% CI] (%)Real-robot taskSuccess rate [95% CI] (%)
Lift Bottle100 [96.3–100.0]Insert Plug70 [48.1–85.5]
Pull-out Key99 [94.6–99.8]Unscrew Cap85 [64.0–94.8]
Lift Can95 [88.8–97.8]Lift Tube95 [76.4–99.1]
Put Bottle90 [82.6–94.5]Upright Bottle90 [69.9–97.2]
Insert Hole98 [93.0–99.4]Wipe Whiteboard95 [76.4–99.1]
Insert HDMI44 [34.7–53.8]Pour Water90 [69.9–97.2]
Insert Tube95 [88.8–97.8]——
Grasp Classify99 [94.6–99.8]——
8-task mean90.0 [87.8–91.5]6-task mean87.5 [79.3–91.1]

Q3. Does the trained assistant retain the ability to predict future tactile changes?

We evaluated the trained online assistant using the same prediction error as the frozen teacher (Appendix E.2). Both models received only current observations. We compared their predictive tokens, before the action readout, with features extracted from the observed tactile image change ten frames later (about 0.33 s). We averaged cosine distance over valid tokens within each frame, then over frames within each task, giving all eight tasks equal weight.

Prediction error on the original teacher-training holdout (lower is better). These episodes were held out from teacher fitting but had been seen during earlier representation learning.

Task or evaluation subsetFrozen teacherTrained online assistant
Grasp Classify0.03720.0454
Insert HDMI0.02490.0560
Insert Hole0.02540.0464
Insert Tube0.05260.0783
Lift Bottle0.02990.0412
Lift Can0.02940.0332
Pull-out Key0.04240.0529
Put Bottle0.04350.0585
Eight-task mean (all valid frames)0.03570.0515
Eight-task mean (image-change subset)0.03710.0502

The assistant had higher error than the teacher on every task, but retained predictive information after action training. On a separate set of episodes withheld from all training stages, the assistant’s mean error was 0.0752, compared with 0.0669 for the teacher and 0.1731 for a predictor that always outputs the training-target mean. On frames with measurable tactile change, the corresponding errors were 0.0933, 0.0646, and 0.1465.

Reviewer_dnGQOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper proposes Teacher-Assistant Distillation (TAD), a method for transferring predictions of future tactile change into a vision-language-action (VLA) policy built on pi0.5. It works in several pieces: frozen teacher, assistant (stage II), asymmetric attention mask, spatial readout.

Critical strengths
  1. Strong empirical results, especially on the real robot. The evaluation covers six physical tasks at 20 trials per task, and TAD outperforms both baselines on every task.
  2. The asymmetric mask closes indirect paths, that mitigates any leaks that VLAs are known to cause from demonstration-action information into "predictive" features through the noisy action input.
  3. Real-world ablations are more rigorous than is typical.
Critical weaknesses

Here are my main concerns with the paper:

  1. All baseline numbers are taken from the N0-VTLA paper rather than re-run.
  2. The asymmetric attention constraint (contribution 3) and the Spatial Readout have no ablation.
  3. The "w/o tactile information" row in Table 3 (30/40/65/45/60/45) matches pi0.5 in Table 2 exactly on all six tasks.
Questions for authors
  1. Was the "w/o tactile information" variant retrained, or are the π0.5 numbers reused? Please explain the exact task-by-task match with Table 2.
  2. Can you report a same-budget tactile-conditioned pi0.5 baseline in simulation, along with simulation ablations of the main components?
  3. Can you ablate the asymmetric attention mask and the Spatial Readout separately?
  4. Can you confirm if the baselines we rerun to report the numbers?
6 comments

Thank you for highlighting the consistency of the real-robot results and for asking us to clarify the baseline comparisons. We address each question below.

Q1. Was the “w/o tactile information” variant retrained, or were the π₀.₅ results reused?

Our base VLA policy is π₀.₅. TAD adds two tactile pathways to this policy. One feeds current tactile features directly into the policy. The other uses teacher–assistant distillation to incorporate predictions of future tactile changes into action generation. Removing both pathways leaves the base π₀.₅ policy. This is what “w/o tactile information” denotes in Table 3.

We therefore reused the π₀.₅ results from Table 2 without retraining the model or running additional robot trials. Both rows report the same 20 trials per task, which explains the exact match across all six tasks.

Q2. How does TAD compare with a same-budget tactile-conditioned π₀.₅ baseline and with ablations of its main components in simulation?

π₀.₅ + current touch and full TAD used the same demonstrations, the same number of policy updates, and the same effective batch size. Both policies receive current tactile observations. The baseline learns from demonstrated actions alone. Full TAD also uses a teacher and an assistant to supervise predictions of future tactile changes. The matched budget covers policy training. TAD’s teacher pretraining incurs additional compute.

Table R1. Tactile supervision and success rates (%) on UniVTAC. Each policy was evaluated for 100 episodes per task. The mean weights all eight tasks equally. Full TAD reproduces submitted Table 1.

TaskFull TADπ₀.₅ + current touchTAD w/o assistantTAD with current targets
Tactile supervisionFuture change, teacher + assistantNone (action loss only)Future change, frozen teacher onlyCurrent tactile features, teacher + assistant
Lift Bottle1001009999
Pull-out Key99829073
Lift Can95838078
Put Bottle90708372
Insert Hole98909588
Insert HDMI44243226
Insert Tube95949992
Grasp Classify9910010099
Mean90.0080.3884.7578.38
  • Current touch as input. Full TAD achieved 90.00% mean success versus 80.38% for π₀.₅ + current touch, a gain of 9.6 percentage points. It improved on six of eight tasks. The largest gains were on Put Bottle (70% to 90%) and Insert HDMI (24% to 44%).
  • Assistant. Keeping future-change supervision but distilling directly from the frozen teacher yielded 84.75%. The 5.25-point difference measures the contribution of the assistant as a whole in this comparison.
  • Future versus current targets. With current touch as input and the teacher–assistant architecture retained, replacing the future-change target with current tactile features reduced mean success from 90.00% to 78.38%. Pull-out Key showed the largest drop (99% to 73%). This comparison tests what the policy is taught to predict while keeping the tactile input fixed.

Q3. Can the asymmetric attention constraint and Spatial Readout be ablated separately?

We ablated one component at a time.

  1. Asymmetric attention constraint. In full TAD, action tokens can read predictive tokens. We relaxed the mask to also let predictive tokens read noisy action tokens. Spatial Readout remained active.
  2. Spatial Readout. We set the spatial update in Eq. 6 to zero, removing the student action states’ direct read from current spatial tactile memory. The asymmetric mask, compressed tactile tokens, and the assistant’s action residual remained active.

Table R2. UniVTAC success rates (%). Each policy was evaluated for 100 episodes per task. The mean weights all eight tasks equally.

TaskFull TADTAD w/o asymmetric attention constraintTAD w/o Spatial Readout
Lift Bottle100100100
Pull-out Key999392
Lift Can958096
Put Bottle908692
Insert Hole988197
Insert HDMI442530
Insert Tube958490
Grasp Classify998099
Mean90.0078.6387.00
Mean drop from full TAD (percentage points)—11.43.0

Relaxing the attention constraint had the larger effect on mean success. Seven of eight tasks declined, with 19-point drops on both Insert HDMI and Grasp Classify. This supports blocking predictive tokens from reading noisy action inputs, which carry information about the demonstrated action during training. The effect of Spatial Readout varied by task. Removing it produced the largest drop on Insert HDMI (44% to 30%), while Lift Can and Put Bottle improved by one and two points, respectively.

Comments on the paper

3 comments

We thank all reviewers for their careful evaluation and constructive feedback. We have redrawn Figure 2 to highlight TAD’s main contribution: a frozen teacher trains the policy to anticipate changes in contact, and a teacher-initialized assistant learns from demonstrated actions to use those predictions to refine control. Our additional ablations show that mean success across eight UniVTAC tasks increased from 84.75% with direct teacher distillation to 87.75% when the assistant was added with the teacher target held fixed, and to 90.00% when the assistant provided adaptive distillation targets. We submitted the revised manuscript on the final day to allow time to complete and verify these experiments. We will incorporate all ablation results reported in the rebuttal into the final manuscript and welcome further discussion.

Thank you all for your careful reviews and thoughtful feedback. With approximately four hours remaining in the discussion period, we welcome any further questions about the revised manuscript or the additional experiments. If any point would benefit from further discussion, please let us know here. We will be available to respond until the deadline.

Dear all,

TAD teaches a robot policy to anticipate changes in contact and use those predictions to guide its actions. It achieved 90.0% mean success across eight UniVTAC tasks and 87.5% across six real-robot tasks. We summarize the revisions and experiments that address each reviewer’s main concern.

dnGQ — Fairness of the baseline comparison. We clarified that the five baseline rows in Table 1 come from the N₀-VTLA paper and that our evaluation used the same UniVTAC protocol and seeds. We also trained a tactile-conditioned π₀.₅ baseline using the same demonstrations, number of policy updates, and effective batch size as TAD. Mean success was 80.38% for this baseline and 90.00% for TAD. In their follow-up, dnGQ supported acceptance and described the remaining baseline concern as non-blocking.

JFcA — Where the gains occur and whether the assistant retains predictive ability. We reported per-task results and confidence intervals. The largest gains over N₀-VTLA were on Put Bottle (60% to 90%) and Insert HDMI (25% to 44%). TAD maintained 95–100% success on each of the other six tasks. On trajectories withheld from every training stage, the assistant predicted future tactile changes with a mean cosine distance of 0.0752, compared with 0.0669 for the frozen teacher and 0.1731 for the mean-target baseline.

efYx — Clarity of Figure 2 and the assistant’s contribution. We redrew Figure 2 in the submitted revision, removing implementation details to show the two stages. The frozen teacher first teaches the policy to predict future tactile changes. The teacher-initialized assistant then adapts through action supervision. Its moving-average copy supplies the student’s training target, while the online assistant corrects actions at deployment. Direct teacher distillation achieved 84.75% mean success. Adding the assistant and its action branch with the teacher target fixed raised success to 87.75%. Switching only to the adaptive target raised it to 90.00%.

We thank the reviewers for their feedback and will include the additional ablations and held-out evaluations in the final manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.