- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper targets a real inefficiency in tactile-conditioned vision-language-action policies: a chunk-based policy conditions the entire action chunk on the tactile observation available before execution begins, so the contact signal is stale for most of the chunk. §2.3 illustrates this with a single dropper-squeezing episode, measuring cosine drift of visual and tactile representations over a 40-action horizon and reporting roughly 0.55 for touch versus 0.005 for vision at the end of the horizon.
The proposed fix, TacForcing, keeps the backbone (π0.5 in simulation, GR00T N1.7 on the robots) but replaces the action expert with a streaming one. The chunk is split into K blocks of B actions; each block k is assigned an individual flow time λ_k(n) = min(n/(kS), 1), so blocks complete at staggered sampling steps and become executable one after another. Completed blocks are dispatched; the partially denoised states of unfinished blocks are retained and further refined after each block executes, conditioned on the tactile observation refreshed at that point. The second component, Execution-Aware Tactile Attention (EATA), adds an additive mask so that the current tactile tokens are directly visible only to the block about to execute, on the argument that later blocks will be executed against a contact state that no longer matches the current reading. Training mirrors inference: a progress p is sampled, block states are constructed from a per-block interpolation, the matching tactile frame and mask are selected, and the loss is applied only to unfinished action positions.
Validation is six UniVTAC simulation tasks (100 rollouts each) and six real-world tasks across two platforms (16 trials each), against vision-only (π0.5, GR00T N1.7), tactile-conditioned (UniVTAC-ACT, FTP-1) and tactile-reactive (RDP, T-Rex) baselines. Reported averages are 65% in simulation (+6 over FTP-1) and 66% on hardware (+15 over FTP-1 and T-Rex). An ablation on three tasks per setting decomposes the gain into tactile refresh and EATA.
Critical strengths
1. The diagnosis is measured rather than assumed, and the method follows from it. Most tactile-VLA papers assert that contact states change faster than vision and then bolt on a high-frequency pathway. This one actually quantifies the drift inside a single chunk (§2.3, Fig. 2) and then makes two design decisions that are direct consequences: near-term actions should be finished and shipped early (block-wise flow scheduling), and only the block that is about to execute should be allowed to see the newest tactile token (EATA). The ablation is decomposed along the same axis — fixed tactile, then refresh, then refresh plus gating — and the refresh step never hurts (39→39 on Pull-out Key, up elsewhere) while the gating step improves all six tasks shown. The largest effects land on exactly the tasks where sustained or changing contact dominates (Insert Hole 34→58→69, Wipe Board 31→44→75). A motivation-mechanism-ablation chain that lines up this cleanly is rarer than it should be, and it is the main reason I am willing to take the method seriously despite the concerns below.
2. The hardware evaluation is substantive, and one detail in the setup makes the main claim cleaner than the authors seem to realise. Six real tasks on two different embodiments (Sharpa North, and a bimanual dual-arm RealMan setup carrying two 22-DoF Sharpa Wave hands), 16 trials each, four baselines including two tactile-reactive ones, and a jump from 45% to 66% over the very backbone the model is initialized from. More interestingly, the tactile encoder is initialized from FTP-1 — a baseline the method beats. That means the +6 in simulation is not a better tactile representation winning; the representation is the same one FTP-1 uses, so the gain is attributable to when and to whom tactile is injected. That is a stronger piece of evidence than the paper makes it out to be, and it is the part of the contribution I would expect others to build on.
Critical weaknesses
1. The headline numbers are quoted at a precision the protocol cannot support, and the aggregate framing overstates the margin. Sixteen trials means a resolution of 6.25 points, and nothing in the paper reports variance, confidence intervals, training seeds, or a significance test. Concretely: the claim that TacForcing "exceeds the best baseline by 6 percentage points on each of the three Sharpa North tasks" rests on 50→56, 75→81 and 44→50, i.e. three independent one-trial differences. The simulation deltas against FTP-1 are +1, +13, −3, +20, +7, +3, so the "first or tied for first on five of six tasks" claim is carried partly by +1 on Lift Bottle (89→90) and +2 on Put Bottle in Shelf, neither of which is distinguishable from sampling noise at 100 rollouts. The aggregate framing compounds this. "Outperforming the strongest baselines by 6 and 15 points" compares against the average of one baseline family; the per-task best baseline in Table 2 is (50, 75, 44, 31, 75, 75), which averages 58.3, so against a per-task oracle the real margin is about 7 points, not 15. In simulation the same computation gives 63.2 for the per-task best baseline against 65.3 for TacForcing, i.e. about +2. None of this means the method doesn't help; the ablation trends say it does. But the abstract claims more than the experiments establish, and people will go on citing "15 points" as if it were a controlled margin.
2. The central mechanism is asserted rather than demonstrated, and the two controls that would justify the architecture are absent. Three things, roughly in the order they bother me.
(a) EATA may not be the barrier it is described as. The mask is applied only to action-to-tactile attention, and the paper states explicitly that the expert's remaining attention structure is preserved. The unfinished blocks are refined jointly by one action expert, so unless action tokens are forbidden from attending to one another, the newest tactile token reaches block k+2 through the block-k action tokens in the same forward pass — one hop, and the mask is bypassed. I assume the action tokens do attend to each other, since that is how π0/GR00T-style action experts are built, but the paper never says and never checks. A one-paragraph diagnostic would settle it: perturb Z, measure how much block k+2's predicted actions move with and without the mask. Without that, EATA is a plausible inductive bias whose mechanism is unverified, and the success-rate ablation cannot separate "the model genuinely respects the temporal barrier" from "the mask happened to regularize training." A cross-ablation (train with mask / infer without, and the reverse) would separate those two readings.
(b) The control that would justify retaining intermediate states is missing. The natural competitor is receding-horizon replanning: call the chunk policy again every B steps with the fresh tactile observation, starting from noise. Practitioners do exactly this, it is trivially cheap, and it does not require any of the streaming machinery. Streaming with retained states is only worth its complexity if restarting from noise every block is materially worse — and the paper does not show that. Note that arm (ii) of the ablation ("tactile updates without EATA") is not this baseline: it retains streaming generation and changes only the tactile input. This matters for the framing too, because B = 5 actions at 30 Hz gives an effective tactile reaction interval of about 167 ms, roughly 6 Hz; that is the number that should be compared against the reactive baselines' control rates, and the same cadence is exactly what a naive replanning baseline would achieve.
(c) There is an uncontrolled compute confound and no latency measurement. In simulation the sampler performs N = KS = 50 sequential integration steps per chunk. A π0/π0.5 action expert typically uses around ten — the paper does not report the baselines' step counts, which is part of the problem. Part of the gap to π0.5 could simply be a five-times larger sampling budget rather than tactile conditioning. The real-world configuration is the reverse worry: with S = 2, block 1, the only block executed immediately, is produced by two Euler steps of size 0.5, while block 8 has accumulated 16 steps. The executed actions are the least-refined ones, and no analysis addresses this. Given that the paper's selling point is removing the separate high-frequency controller, the complete absence of wall-clock data is conspicuous: no per-block inference latency, no achieved control frequency, no statement of whether generation runs concurrently with execution, no GPU. "Lower architectural complexity" is plausible but currently unmeasured — the complexity may simply have moved into the inference schedule.
Separately, and shorter: the motivating measurement in Fig. 2 compares cosine distances across two unrelated embedding spaces produced by two differently-trained encoders, one of which (the tactile encoder) is taken from the trained TacForcing model itself (Appendix A.3), i.e. it has been optimised to be contact-sensitive, while the visual encoder is frozen. A near-zero visual drift may say more about a pooled frozen visual token than about the episode. The evidence is one episode on one task.
Questions for authors
-
(Affects score) What is the per-task-oracle comparison? Please report, for each table, TacForcing against the best baseline per task rather than the best baseline on average, plus binomial confidence intervals for the 16-trial results. How many training seeds? How is success defined on the real tasks, and are initial object poses randomised across the 16 trials?
-
(Affects score) Why are some baselines so low? FTP-1 scores 6% on Transfer Liquid — the dropper-squeezing task that Fig. 2 uses as the motivating example — and UniVTAC-ACT scores 6% on Lift Can. Both look like under-tuned baselines rather than faithful reproductions. Please state the provenance of each baseline (released checkpoint vs. retrained), the data each was trained or fine-tuned on, the number of configurations tried, and whether any per-task tuning was done. If FTP-1 and UniVTAC-ACT use released generalist checkpoints while π0.5 and RDP were retrained on 50 task-specific demonstrations, the comparison is not obviously like-for-like in either direction.
-
(Affects score) Please add the receding-horizon replanning baseline: same backbone, same tactile encoder, re-invoke the chunk policy every 5 steps from fresh noise. This is the simplest thing that uses execution-time tactile, and without it I cannot tell whether retained intermediate states earn their complexity.
-
(Affects score) Please report wall-clock numbers: per-block inference latency, achieved control frequency, whether denoising overlaps with execution, GPU model, and a step-matched variant of the chunk policy using N = 50 (simulation) for a fair comparison.
-
(Affects score) Do action tokens attend to one another across blocks? If so, how is the EATA restriction enforced beyond the single attention edge? A perturbation diagnostic on Z (how much does block k+2's prediction change with and without the mask?) plus a train-with-mask/infer-without cross-ablation would resolve both the mechanism question and the "regularisation" alternative.
-
What are the sensitivities to B, K and S? B = 5 is used throughout, but B is precisely the tactile refresh interval, so it is the parameter that should be swept. What happens at B = H (no streaming) and at B = 1 (maximal refresh, presumably latency-bound)? Likewise, is S = 2 in the real world a deliberate choice, or the largest value that kept up with the control loop?
-
Regarding Fig. 2: could you show a within-modality temporal drift, or better, a predictive measure — e.g. how much does the next block's action accuracy degrade when conditioned on a tactile token from t − 5B versus t? A same-encoder control would also help, since 0.005 and 0.55 are not on a common scale.
-
The language instruction never varies within a task, so vision-language generalisation is not exercised anywhere in the paper. Is there any evidence (unseen objects, paraphrased instructions, a held-out scene) that the VLA components contribute at all, or is the system effectively a tactile-conditioned imitation policy?
Minor points, for revision only, no response needed: the bold/underline convention in Table 2's caption breaks on Wipe Board, where three methods tie at 75; the Fig. 2 inset legend says "Ego view / Right wrist view" while the caption and §2.3 say "top and wrist cameras"; Appendix B.1 describes the Sharpa & RealMan hardware in detail but never describes Sharpa North; and Figure 5 covers three of six tasks per setting without saying how that subset was chosen — a sentence on the selection rule (and ideally the full six-task version) would prevent readers from wondering whether the third tasks were the difficult ones.
Where this leaves me: I am at weak accept, and that is a genuine borderline rather than a hedge. The idea is worth having in the literature and the real-robot work is expensive and well done, so I would rather see it published with the numbers fixed than rejected for framing. But if items 1, 3 and 4 come back showing that the margin over a per-task oracle is within noise and that plain replanning every five steps does as well, I would move down rather than up, and I think the AC should weight the discussion accordingly.
5 comments
Response to Reviewer VifA (2/4): Replanning, Sampling Budget and Latency
Q3 & W2(b). Comparison with receding-horizon replanning
“Please add the receding-horizon replanning baseline: same backbone, same tactile encoder, re-invoke the chunk policy every 5 steps from noise.”
Thank you for this suggestion. We have added this comparison to Section 4.4, Table 4. We separately train a conventional flow-matching baseline with the same backbone, tactile encoder initialization, data, and training hyperparameters. It restarts from noise every B = 5 executed actions using fresh tactile feedback and matched sampling steps per update (S = 5 in simulation, S = 2 on Sharpa North).
We evaluate variants with and without cached VLM features. The cached variant has inference and deployment latencies comparable to TacForcing. Recomputing VLM features at every update adds approximately 215 ms of GPU computation per 40 actions on Sharpa North (7 × 30.77 ms). Runtime measurements are provided under Q4 below.
| Method | Pull-out Key (%) | Lift Can (%) | Insert Hole (%) | Close Cap | Insert Battery | Close Lid |
|---|---|---|---|---|---|---|
| Replanning | 16 | 43 | 43 | 5/16 | 9/16 | 4/16 |
| Replanning + cached VLM | 20 | 39 | 36 | 7/16 | 6/16 | 5/16 |
| TacForcing | 48 | 63 | 69 | 9/16 | 13/16 | 8/16 |
Simulation: 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.
TacForcing achieves higher observed success rates on all six tasks, supporting the value of retaining intermediate action states. We also observe greater jitter under frequent replanning on hardware, which may reflect reduced temporal consistency from repeatedly restarting generation.
Q4 & W2(c). Sampling budget and deployment latency
(a) Matched sampling steps in simulation
“Part of the gap to π0.5 could simply be a five-times larger sampling budget rather than tactile conditioning.”
“A step-matched variant of the chunk policy using N = 50 (simulation) for a fair comparison.”
The reported π₀.₅ baseline already uses N = 50 sampling steps per chunk, as do TacForcing and FTP-1. These comparisons therefore match the total number of sampling steps. We have clarified the baseline training and inference settings in Appendix A.1.
(b) Real-world inference latency and execution frequency
“Please report wall-clock numbers: per-block inference latency, achieved control frequency, whether denoising overlaps with execution, GPU model…”
We have added these measurements to Section 4.4, Table 3, with component timings in Appendix A.3. Measurements cover 96 blocks after warm-up on Sharpa North using an RTX 4090 (24 GB), with B = 5, S = 2, and 60 Hz action execution within blocks. Inference and execution do not overlap, and no additional inference acceleration is applied.
| Metric | Result |
|---|---|
| First-block latency (VLM + tactile + S steps) | 60.0 ms |
| Subsequent-block latency (tactile + S steps) | 14.8 ms |
| Effective action rate | 48.3 Hz |
| Tactile update rate | 9.66 Hz |
| Tactile update rate (excluding action execution) | 50.3 Hz |
Mean latencies include preprocessing, computation, and command dispatch. “First block” refers to each chunk’s first block, “tactile” denotes tactile encoding, and S denotes the number of flow-matching steps per update (S = 2). The 48.3/9.66 Hz rates include execution and inference pauses. Removing the measured 80.8% execution-time share gives 9.66/(1 − 0.808) ≈ 50.3 Hz, a processing-only rate.
Response to Reviewer VifA (3/4): EATA Mechanism and Ablations
Q5 & W2(a). EATA mechanism and control experiments
(a) Indirect tactile propagation through action tokens
“Do action tokens attend to one another across blocks? If so, how is the EATA restriction enforced beyond the single attention edge?”
Thank you for highlighting this point. Action tokens retain bidirectional attention across blocks for joint refinement, so tactile information can propagate indirectly. EATA restricts direct action-to-tactile attention and does not enforce complete temporal isolation. We clarify this in the paper and examine its practical effect below.
(b) Perturbation analysis
“A perturbation diagnostic on Z (how much does block k+2's prediction change with and without the mask)”
To simulate realistic tactile variations, we replace the current observation with a recorded frame from another trajectory of the same task, rather than synthetic noise. We analyze 3,757 chunks from 60 trajectories across Close Cap, Close Lid, and Insert Battery. Using the same EATA-trained checkpoint for each task with EATA enabled or disabled, we hold intermediate action states and other inputs fixed. Within each mask setting, we measure the future block’s RMS change in flow-velocity prediction relative to the current block’s change in a single forward pass.
| Future-block distance | EATA enabled | EATA disabled |
|---|---|---|
| d = 1 | 18.7 [18.1, 19.3] | 91.8 [91.5, 92.2] |
| d = 2 | 13.5 [13.2, 13.9] | 88.5 [88.0, 89.2] |
| d = 7 | 5.4 [5.2, 5.6] | 85.4 [84.4, 86.6] |
Median response ratios (%) with 95% episode-bootstrap confidence intervals.
EATA reduces the relative response of future blocks despite indirect propagation. This measures sensitivity within one forward pass, not cumulative changes in final actions. Details are provided in Appendix F, Table 11.
(c) Training and inference cross-ablation
“A cross-ablation (train with mask / infer without, and the reverse) would separate those two readings.”
We evaluate models trained with and without EATA under both inference settings, keeping each model’s weights fixed. All conditions retain streaming generation and block-level tactile updates (Appendix E, Table 10).
| Training | Inference | Pull-out Key | Lift Can | Insert Hole | Avg. |
|---|---|---|---|---|---|
| With EATA | With EATA | 48 | 63 | 69 | 60.0 |
| With EATA | Without EATA | 40 | 59 | 59 | 52.7 |
| Without EATA | With EATA | 37 | 43 | 60 | 46.7 |
| Without EATA | Without EATA | 39 | 55 | 58 | 50.7 |
Success rates (%) over 100 simulation trials per task. Avg. is the mean across the three tasks.
Removing EATA at inference lowers average success from 60.0% to 52.7%, while adding it only at inference does not improve average performance. The crossed settings introduce training–inference mismatch, so they cannot alone rule out a regularization effect. We therefore further examine the source of EATA’s gains using consistently trained and evaluated variants below.
(d) Temporal alignment beyond attention restriction
We separately retrain two variants with otherwise identical training and inference settings. Delayed feedback preserves the same EATA mask but uses tactile observations from two blocks earlier. Two-block conditioning lets the latest observation directly condition the next two blocks. Each rule is applied consistently during training and inference (Section 4.4, Table 4).
| Method | Pull-out Key (%) | Lift Can (%) | Insert Hole (%) | Close Cap | Insert Battery | Close Lid |
|---|---|---|---|---|---|---|
| Delayed feedback | 38 | 52 | 53 | 4/16 | 8/16 | 2/16 |
| Two-block conditioning | 41 | 61 | 62 | 6/16 | 10/16 | 4/16 |
| TacForcing | 48 | 63 | 69 | 9/16 | 13/16 | 8/16 |
Simulation: 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.
Both variants achieve lower observed success rates on all six tasks. In particular, the decline with delayed feedback under an unchanged mask supports the importance of aligning tactile observations with action execution beyond attention restriction alone.
Response to Reviewer VifA (1/4): Evaluation Reliability and Baselines
Thank you for your detailed review and constructive questions. We address your concerns in four parts.
Q1 & W1. Statistical reliability and per-task comparisons
We have revised the Abstract, Introduction, Results, and Conclusion, replacing the broad “6 and 15 percentage points” claims with observed success rates and more cautious comparisons. We conducted 16 additional trials per method on each real-world task, bringing the total to 32. The main results table now reports successful trials out of 32, making differences in success counts explicit.
(a) Comparison with the best baseline per task
The requested comparisons are included in Appendix D. Below, we report the best observed baseline on each task and 95% Wilson confidence intervals for the expanded real-world evaluation. Simulation retains the original 100 trials per task.
Simulation: success rates (%).
Real world: success rates (%) [95% Wilson CI].
Averages weight the six tasks equally. Δ is the observed difference in percentage points, calculated before rounding. Real-world intervals use 32 trials per method per task. Observed differences do not imply statistical significance.
(b) Training seeds and evaluation protocol
Our training uses one seed (42), and we evaluate the checkpoint from the final training step. Initial object positions vary across four predefined configurations shared across methods for fairness, with eight trials per configuration. They are not randomly sampled per trial. Training settings and task-specific success/failure criteria are documented in Appendices A and B.2.
Q2. Baseline configurations and failure analysis
We have added training and inference settings to Appendices A.1 and A.2. All evaluated models are task-specific fine-tuned checkpoints. In simulation, FTP-1 and UniVTAC-ACT use the authors’ officially released task-specific checkpoints. TacForcing and the remaining baselines are trained on the same task-specific datasets. For our training runs, we follow the official implementations and recommended hyperparameters, using one configuration per method per setting across tasks, without additional searches or task-specific tuning.
For FTP-1 on Transfer Liquid, we frequently observe dropper slippage or liquid leakage during transfer. Its lower tactile update frequency may limit timely grip adjustments as contact conditions change. For UniVTAC-ACT, the reported 6% corresponds to Put Bottle in Shelf, while Lift Can achieves 24%. Put Bottle in Shelf requires the largest arm movements among the simulation tasks, and we observe arm oscillations that may hinder placement. Task snapshots are provided in Appendix C.