acceptodds
Under review as a conference paper at ICLR 2027

TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback

Abstract

Contact-rich manipulation requires adapting to contact states that can evolve substantially within an action horizon. However, chunk-based vision-language-action models predict complete action chunks from observations collected before execution, leaving tactile conditioning stale during execution. Existing tactile-reactive approaches typically rely on separate high-frequency controllers, which increase both architectural and training complexity. In this paper, we introduce TacForcing, a streaming action-generation framework that effectively incorporates execution-time tactile feedback. TacForcing replaces the standard action expert with a streaming expert that generates action blocks sequentially while preserving intermediate states of unfinished blocks. After each block is executed, the expert resumes generation from these states using newly acquired tactile feedback. To better align tactile conditioning with action execution, we further introduce Execution-Aware Tactile Attention (EATA), which restricts direct access to each tactile update to the next block scheduled for execution. Across six UniVTAC simulation tasks and six real-world contact-rich manipulation tasks on two robot platforms, TacForcing achieves average success rates of 65% in simulation and 66% in the real world, outperforming the strongest baselines by 6 and 15 percentage points, respectively.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, vision-language-action models, tactile feedback, contact-rich manipulation, streaming action generation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_VifAOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets a real inefficiency in tactile-conditioned vision-language-action policies: a chunk-based policy conditions the entire action chunk on the tactile observation available before execution begins, so the contact signal is stale for most of the chunk. §2.3 illustrates this with a single dropper-squeezing episode, measuring cosine drift of visual and tactile representations over a 40-action horizon and reporting roughly 0.55 for touch versus 0.005 for vision at the end of the horizon.

The proposed fix, TacForcing, keeps the backbone (π0.5 in simulation, GR00T N1.7 on the robots) but replaces the action expert with a streaming one. The chunk is split into K blocks of B actions; each block k is assigned an individual flow time λ_k(n) = min(n/(kS), 1), so blocks complete at staggered sampling steps and become executable one after another. Completed blocks are dispatched; the partially denoised states of unfinished blocks are retained and further refined after each block executes, conditioned on the tactile observation refreshed at that point. The second component, Execution-Aware Tactile Attention (EATA), adds an additive mask so that the current tactile tokens are directly visible only to the block about to execute, on the argument that later blocks will be executed against a contact state that no longer matches the current reading. Training mirrors inference: a progress p is sampled, block states are constructed from a per-block interpolation, the matching tactile frame and mask are selected, and the loss is applied only to unfinished action positions.

Validation is six UniVTAC simulation tasks (100 rollouts each) and six real-world tasks across two platforms (16 trials each), against vision-only (π0.5, GR00T N1.7), tactile-conditioned (UniVTAC-ACT, FTP-1) and tactile-reactive (RDP, T-Rex) baselines. Reported averages are 65% in simulation (+6 over FTP-1) and 66% on hardware (+15 over FTP-1 and T-Rex). An ablation on three tasks per setting decomposes the gain into tactile refresh and EATA.

Critical strengths

1. The diagnosis is measured rather than assumed, and the method follows from it. Most tactile-VLA papers assert that contact states change faster than vision and then bolt on a high-frequency pathway. This one actually quantifies the drift inside a single chunk (§2.3, Fig. 2) and then makes two design decisions that are direct consequences: near-term actions should be finished and shipped early (block-wise flow scheduling), and only the block that is about to execute should be allowed to see the newest tactile token (EATA). The ablation is decomposed along the same axis — fixed tactile, then refresh, then refresh plus gating — and the refresh step never hurts (39→39 on Pull-out Key, up elsewhere) while the gating step improves all six tasks shown. The largest effects land on exactly the tasks where sustained or changing contact dominates (Insert Hole 34→58→69, Wipe Board 31→44→75). A motivation-mechanism-ablation chain that lines up this cleanly is rarer than it should be, and it is the main reason I am willing to take the method seriously despite the concerns below.

2. The hardware evaluation is substantive, and one detail in the setup makes the main claim cleaner than the authors seem to realise. Six real tasks on two different embodiments (Sharpa North, and a bimanual dual-arm RealMan setup carrying two 22-DoF Sharpa Wave hands), 16 trials each, four baselines including two tactile-reactive ones, and a jump from 45% to 66% over the very backbone the model is initialized from. More interestingly, the tactile encoder is initialized from FTP-1 — a baseline the method beats. That means the +6 in simulation is not a better tactile representation winning; the representation is the same one FTP-1 uses, so the gain is attributable to when and to whom tactile is injected. That is a stronger piece of evidence than the paper makes it out to be, and it is the part of the contribution I would expect others to build on.

Critical weaknesses

1. The headline numbers are quoted at a precision the protocol cannot support, and the aggregate framing overstates the margin. Sixteen trials means a resolution of 6.25 points, and nothing in the paper reports variance, confidence intervals, training seeds, or a significance test. Concretely: the claim that TacForcing "exceeds the best baseline by 6 percentage points on each of the three Sharpa North tasks" rests on 50→56, 75→81 and 44→50, i.e. three independent one-trial differences. The simulation deltas against FTP-1 are +1, +13, −3, +20, +7, +3, so the "first or tied for first on five of six tasks" claim is carried partly by +1 on Lift Bottle (89→90) and +2 on Put Bottle in Shelf, neither of which is distinguishable from sampling noise at 100 rollouts. The aggregate framing compounds this. "Outperforming the strongest baselines by 6 and 15 points" compares against the average of one baseline family; the per-task best baseline in Table 2 is (50, 75, 44, 31, 75, 75), which averages 58.3, so against a per-task oracle the real margin is about 7 points, not 15. In simulation the same computation gives 63.2 for the per-task best baseline against 65.3 for TacForcing, i.e. about +2. None of this means the method doesn't help; the ablation trends say it does. But the abstract claims more than the experiments establish, and people will go on citing "15 points" as if it were a controlled margin.

2. The central mechanism is asserted rather than demonstrated, and the two controls that would justify the architecture are absent. Three things, roughly in the order they bother me.

(a) EATA may not be the barrier it is described as. The mask is applied only to action-to-tactile attention, and the paper states explicitly that the expert's remaining attention structure is preserved. The unfinished blocks are refined jointly by one action expert, so unless action tokens are forbidden from attending to one another, the newest tactile token reaches block k+2 through the block-k action tokens in the same forward pass — one hop, and the mask is bypassed. I assume the action tokens do attend to each other, since that is how π0/GR00T-style action experts are built, but the paper never says and never checks. A one-paragraph diagnostic would settle it: perturb Z, measure how much block k+2's predicted actions move with and without the mask. Without that, EATA is a plausible inductive bias whose mechanism is unverified, and the success-rate ablation cannot separate "the model genuinely respects the temporal barrier" from "the mask happened to regularize training." A cross-ablation (train with mask / infer without, and the reverse) would separate those two readings.

(b) The control that would justify retaining intermediate states is missing. The natural competitor is receding-horizon replanning: call the chunk policy again every B steps with the fresh tactile observation, starting from noise. Practitioners do exactly this, it is trivially cheap, and it does not require any of the streaming machinery. Streaming with retained states is only worth its complexity if restarting from noise every block is materially worse — and the paper does not show that. Note that arm (ii) of the ablation ("tactile updates without EATA") is not this baseline: it retains streaming generation and changes only the tactile input. This matters for the framing too, because B = 5 actions at 30 Hz gives an effective tactile reaction interval of about 167 ms, roughly 6 Hz; that is the number that should be compared against the reactive baselines' control rates, and the same cadence is exactly what a naive replanning baseline would achieve.

(c) There is an uncontrolled compute confound and no latency measurement. In simulation the sampler performs N = KS = 50 sequential integration steps per chunk. A π0/π0.5 action expert typically uses around ten — the paper does not report the baselines' step counts, which is part of the problem. Part of the gap to π0.5 could simply be a five-times larger sampling budget rather than tactile conditioning. The real-world configuration is the reverse worry: with S = 2, block 1, the only block executed immediately, is produced by two Euler steps of size 0.5, while block 8 has accumulated 16 steps. The executed actions are the least-refined ones, and no analysis addresses this. Given that the paper's selling point is removing the separate high-frequency controller, the complete absence of wall-clock data is conspicuous: no per-block inference latency, no achieved control frequency, no statement of whether generation runs concurrently with execution, no GPU. "Lower architectural complexity" is plausible but currently unmeasured — the complexity may simply have moved into the inference schedule.

Separately, and shorter: the motivating measurement in Fig. 2 compares cosine distances across two unrelated embedding spaces produced by two differently-trained encoders, one of which (the tactile encoder) is taken from the trained TacForcing model itself (Appendix A.3), i.e. it has been optimised to be contact-sensitive, while the visual encoder is frozen. A near-zero visual drift may say more about a pooled frozen visual token than about the episode. The evidence is one episode on one task.

Questions for authors
  1. (Affects score) What is the per-task-oracle comparison? Please report, for each table, TacForcing against the best baseline per task rather than the best baseline on average, plus binomial confidence intervals for the 16-trial results. How many training seeds? How is success defined on the real tasks, and are initial object poses randomised across the 16 trials?

  2. (Affects score) Why are some baselines so low? FTP-1 scores 6% on Transfer Liquid — the dropper-squeezing task that Fig. 2 uses as the motivating example — and UniVTAC-ACT scores 6% on Lift Can. Both look like under-tuned baselines rather than faithful reproductions. Please state the provenance of each baseline (released checkpoint vs. retrained), the data each was trained or fine-tuned on, the number of configurations tried, and whether any per-task tuning was done. If FTP-1 and UniVTAC-ACT use released generalist checkpoints while π0.5 and RDP were retrained on 50 task-specific demonstrations, the comparison is not obviously like-for-like in either direction.

  3. (Affects score) Please add the receding-horizon replanning baseline: same backbone, same tactile encoder, re-invoke the chunk policy every 5 steps from fresh noise. This is the simplest thing that uses execution-time tactile, and without it I cannot tell whether retained intermediate states earn their complexity.

  4. (Affects score) Please report wall-clock numbers: per-block inference latency, achieved control frequency, whether denoising overlaps with execution, GPU model, and a step-matched variant of the chunk policy using N = 50 (simulation) for a fair comparison.

  5. (Affects score) Do action tokens attend to one another across blocks? If so, how is the EATA restriction enforced beyond the single attention edge? A perturbation diagnostic on Z (how much does block k+2's prediction change with and without the mask?) plus a train-with-mask/infer-without cross-ablation would resolve both the mechanism question and the "regularisation" alternative.

  6. What are the sensitivities to B, K and S? B = 5 is used throughout, but B is precisely the tactile refresh interval, so it is the parameter that should be swept. What happens at B = H (no streaming) and at B = 1 (maximal refresh, presumably latency-bound)? Likewise, is S = 2 in the real world a deliberate choice, or the largest value that kept up with the control loop?

  7. Regarding Fig. 2: could you show a within-modality temporal drift, or better, a predictive measure — e.g. how much does the next block's action accuracy degrade when conditioned on a tactile token from t − 5B versus t? A same-encoder control would also help, since 0.005 and 0.55 are not on a common scale.

  8. The language instruction never varies within a task, so vision-language generalisation is not exercised anywhere in the paper. Is there any evidence (unseen objects, paraphrased instructions, a held-out scene) that the VLA components contribute at all, or is the system effectively a tactile-conditioned imitation policy?

Minor points, for revision only, no response needed: the bold/underline convention in Table 2's caption breaks on Wipe Board, where three methods tie at 75; the Fig. 2 inset legend says "Ego view / Right wrist view" while the caption and §2.3 say "top and wrist cameras"; Appendix B.1 describes the Sharpa & RealMan hardware in detail but never describes Sharpa North; and Figure 5 covers three of six tasks per setting without saying how that subset was chosen — a sentence on the selection rule (and ideally the full six-task version) would prevent readers from wondering whether the third tasks were the difficult ones.

Where this leaves me: I am at weak accept, and that is a genuine borderline rather than a hedge. The idea is worth having in the literature and the real-robot work is expensive and well done, so I would rather see it published with the numbers fixed than rejected for framing. But if items 1, 3 and 4 come back showing that the margin over a per-task oracle is within noise and that plain replanning every five steps does as well, I would move down rather than up, and I think the AC should weight the discussion accordingly.

5 comments
AuthorsOpenReview ↗

Response to Reviewer VifA (1/4): Evaluation Reliability and Baselines

Thank you for your detailed review and constructive questions. We address your concerns in four parts.

Q1 & W1. Statistical reliability and per-task comparisons

“The headline numbers are quoted at a precision the protocol cannot support, and the aggregate framing overstates the margin.”

We have revised the Abstract, Introduction, Results, and Conclusion, replacing the broad “6 and 15 percentage points” claims with observed success rates and more cautious comparisons. We conducted 16 additional trials per method on each real-world task, bringing the total to 32. The main results table now reports successful trials out of 32, making differences in success counts explicit.

(a) Comparison with the best baseline per task

“Please report, for each table, TacForcing against the best baseline per task rather than the best baseline on average, plus binomial confidence intervals for the 16-trial results.”

The requested comparisons are included in Appendix D. Below, we report the best observed baseline on each task and 95% Wilson confidence intervals for the expanded real-world evaluation. Simulation retains the original 100 trials per task.

Simulation: success rates (%).

TaskBest baselineBaselineTacForcingΔ (pp)
Lift BottleFTP-189.090.0+1.0
Pull-out Keyπ₀.₅43.048.0+5.0
Lift CanFTP-166.063.0−3.0
Put Bottle in Shelfπ₀.₅43.043.00.0
Insert HoleFTP-162.069.0+7.0
Insert TubeFTP-176.079.0+3.0
AveragePer-task oracle63.265.3+2.2

Real world: success rates (%) [95% Wilson CI].

TaskBest baselineBaselineTacForcingΔ (pp)
Close CapT-Rex53.1 [36.4, 69.1]62.5 [45.3, 77.1]+9.4
Insert BatteryGR00T N1.7 / FTP-165.6 [48.3, 79.6]75.0 [57.9, 86.7]+9.4
Close LidT-Rex46.9 [30.9, 63.6]56.3 [39.3, 71.8]+9.4
Transfer LiquidT-Rex28.1 [15.6, 45.4]43.8 [28.2, 60.7]+15.6
Wipe BoardFTP-168.8 [51.4, 82.0]78.1 [61.2, 89.0]+9.4
Stand BottleT-Rex78.1 [61.2, 89.0]81.3 [64.7, 91.1]+3.1
AveragePer-task oracle56.866.1+9.4

Averages weight the six tasks equally. Δ is the observed difference in percentage points, calculated before rounding. Real-world intervals use 32 trials per method per task. Observed differences do not imply statistical significance.

(b) Training seeds and evaluation protocol

Our training uses one seed (42), and we evaluate the checkpoint from the final training step. Initial object positions vary across four predefined configurations shared across methods for fairness, with eight trials per configuration. They are not randomly sampled per trial. Training settings and task-specific success/failure criteria are documented in Appendices A and B.2.

Q2. Baseline configurations and failure analysis

“Please state the provenance of each baseline (released checkpoint vs. retrained), the data each was trained or fine-tuned on, the number of configurations tried, and whether any per-task tuning was done.”

We have added training and inference settings to Appendices A.1 and A.2. All evaluated models are task-specific fine-tuned checkpoints. In simulation, FTP-1 and UniVTAC-ACT use the authors’ officially released task-specific checkpoints. TacForcing and the remaining baselines are trained on the same task-specific datasets. For our training runs, we follow the official implementations and recommended hyperparameters, using one configuration per method per setting across tasks, without additional searches or task-specific tuning.

For FTP-1 on Transfer Liquid, we frequently observe dropper slippage or liquid leakage during transfer. Its lower tactile update frequency may limit timely grip adjustments as contact conditions change. For UniVTAC-ACT, the reported 6% corresponds to Put Bottle in Shelf, while Lift Can achieves 24%. Put Bottle in Shelf requires the largest arm movements among the simulation tasks, and we observe arm oscillations that may hinder placement. Task snapshots are provided in Appendix C.

AuthorsOpenReview ↗

Response to Reviewer VifA (2/4): Replanning, Sampling Budget and Latency

Q3 & W2(b). Comparison with receding-horizon replanning

“Please add the receding-horizon replanning baseline: same backbone, same tactile encoder, re-invoke the chunk policy every 5 steps from noise.”

Thank you for this suggestion. We have added this comparison to Section 4.4, Table 4. We separately train a conventional flow-matching baseline with the same backbone, tactile encoder initialization, data, and training hyperparameters. It restarts from noise every B = 5 executed actions using fresh tactile feedback and matched sampling steps per update (S = 5 in simulation, S = 2 on Sharpa North).

We evaluate variants with and without cached VLM features. The cached variant has inference and deployment latencies comparable to TacForcing. Recomputing VLM features at every update adds approximately 215 ms of GPU computation per 40 actions on Sharpa North (7 × 30.77 ms). Runtime measurements are provided under Q4 below.

MethodPull-out Key (%)Lift Can (%)Insert Hole (%)Close CapInsert BatteryClose Lid
Replanning1643435/169/164/16
Replanning + cached VLM2039367/166/165/16
TacForcing4863699/1613/168/16

Simulation: 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.

TacForcing achieves higher observed success rates on all six tasks, supporting the value of retaining intermediate action states. We also observe greater jitter under frequent replanning on hardware, which may reflect reduced temporal consistency from repeatedly restarting generation.

Q4 & W2(c). Sampling budget and deployment latency

(a) Matched sampling steps in simulation

“Part of the gap to π0.5 could simply be a five-times larger sampling budget rather than tactile conditioning.”

“A step-matched variant of the chunk policy using N = 50 (simulation) for a fair comparison.”

The reported π₀.₅ baseline already uses N = 50 sampling steps per chunk, as do TacForcing and FTP-1. These comparisons therefore match the total number of sampling steps. We have clarified the baseline training and inference settings in Appendix A.1.

(b) Real-world inference latency and execution frequency

“Please report wall-clock numbers: per-block inference latency, achieved control frequency, whether denoising overlaps with execution, GPU model…”

We have added these measurements to Section 4.4, Table 3, with component timings in Appendix A.3. Measurements cover 96 blocks after warm-up on Sharpa North using an RTX 4090 (24 GB), with B = 5, S = 2, and 60 Hz action execution within blocks. Inference and execution do not overlap, and no additional inference acceleration is applied.

MetricResult
First-block latency (VLM + tactile + S steps)60.0 ms
Subsequent-block latency (tactile + S steps)14.8 ms
Effective action rate48.3 Hz
Tactile update rate9.66 Hz
Tactile update rate (excluding action execution)50.3 Hz

Mean latencies include preprocessing, computation, and command dispatch. “First block” refers to each chunk’s first block, “tactile” denotes tactile encoding, and S denotes the number of flow-matching steps per update (S = 2). The 48.3/9.66 Hz rates include execution and inference pauses. Removing the measured 80.8% execution-time share gives 9.66/(1 − 0.808) ≈ 50.3 Hz, a processing-only rate.

AuthorsOpenReview ↗

Response to Reviewer VifA (3/4): EATA Mechanism and Ablations

Q5 & W2(a). EATA mechanism and control experiments

(a) Indirect tactile propagation through action tokens

“Do action tokens attend to one another across blocks? If so, how is the EATA restriction enforced beyond the single attention edge?”

Thank you for highlighting this point. Action tokens retain bidirectional attention across blocks for joint refinement, so tactile information can propagate indirectly. EATA restricts direct action-to-tactile attention and does not enforce complete temporal isolation. We clarify this in the paper and examine its practical effect below.

(b) Perturbation analysis

“A perturbation diagnostic on Z (how much does block k+2's prediction change with and without the mask)”

To simulate realistic tactile variations, we replace the current observation with a recorded frame from another trajectory of the same task, rather than synthetic noise. We analyze 3,757 chunks from 60 trajectories across Close Cap, Close Lid, and Insert Battery. Using the same EATA-trained checkpoint for each task with EATA enabled or disabled, we hold intermediate action states and other inputs fixed. Within each mask setting, we measure the future block’s RMS change in flow-velocity prediction relative to the current block’s change in a single forward pass.

Future-block distanceEATA enabledEATA disabled
d = 118.7 [18.1, 19.3]91.8 [91.5, 92.2]
d = 213.5 [13.2, 13.9]88.5 [88.0, 89.2]
d = 75.4 [5.2, 5.6]85.4 [84.4, 86.6]

Median response ratios (%) with 95% episode-bootstrap confidence intervals.

EATA reduces the relative response of future blocks despite indirect propagation. This measures sensitivity within one forward pass, not cumulative changes in final actions. Details are provided in Appendix F, Table 11.

(c) Training and inference cross-ablation

“A cross-ablation (train with mask / infer without, and the reverse) would separate those two readings.”

We evaluate models trained with and without EATA under both inference settings, keeping each model’s weights fixed. All conditions retain streaming generation and block-level tactile updates (Appendix E, Table 10).

TrainingInferencePull-out KeyLift CanInsert HoleAvg.
With EATAWith EATA48636960.0
With EATAWithout EATA40595952.7
Without EATAWith EATA37436046.7
Without EATAWithout EATA39555850.7

Success rates (%) over 100 simulation trials per task. Avg. is the mean across the three tasks.

Removing EATA at inference lowers average success from 60.0% to 52.7%, while adding it only at inference does not improve average performance. The crossed settings introduce training–inference mismatch, so they cannot alone rule out a regularization effect. We therefore further examine the source of EATA’s gains using consistently trained and evaluated variants below.

(d) Temporal alignment beyond attention restriction

We separately retrain two variants with otherwise identical training and inference settings. Delayed feedback preserves the same EATA mask but uses tactile observations from two blocks earlier. Two-block conditioning lets the latest observation directly condition the next two blocks. Each rule is applied consistently during training and inference (Section 4.4, Table 4).

MethodPull-out Key (%)Lift Can (%)Insert Hole (%)Close CapInsert BatteryClose Lid
Delayed feedback3852534/168/162/16
Two-block conditioning4161626/1610/164/16
TacForcing4863699/1613/168/16

Simulation: 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.

Both variants achieve lower observed success rates on all six tasks. In particular, the decline with delayed feedback under an unchanged mask supports the importance of aligning tactile observations with action execution beyond attention restriction alone.

Reviewer_PpiNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TacForcing, a streaming action-generation framework designed to incorporate execution-time tactile feedback into flow-based vision-language-action policies for contact-rich manipulation.

The proposed method addresses the temporal mismatch between tactile observations and action execution in conventional chunk-based policies. Specifically, TacForcing divides an action chunk into multiple blocks and progressively generates them using block-wise flow scheduling. Completed blocks are executed sequentially, while unfinished blocks retain their intermediate generation states and are further refined using newly acquired tactile observations.

The paper also introduces Execution-Aware Tactile Attention (EATA), which restricts direct access to the latest tactile feedback to the next block scheduled for execution, aiming to improve temporal alignment between tactile conditioning and action execution.

The method is evaluated on six simulated tasks from UniVTAC and six real-world manipulation tasks across two robotic platforms. TacForcing achieves average success rates of 65% in simulation and 66% in real-world experiments, outperforming the strongest reported baselines by 6 and 15 percentage points, respectively. Ablation studies further demonstrate improvements from both execution-time tactile updates and EATA.

Critical strengths
  1. Well-motivated problem and a coherent framework.

The paper addresses an important challenge in contact-rich manipulation: the temporal mismatch between tactile observations and the execution of chunk-based actions. The motivation is clearly presented, and the proposed integration of execution-time tactile feedback into streaming action generation is conceptually reasonable.

The Execution-Aware Tactile Attention mechanism provides an interesting perspective on aligning sensory conditioning with action execution. Moreover, the framework incorporates tactile feedback directly into the action-generation process without requiring a separate high-frequency reactive controller.

  1. Encouraging experimental results across simulation and real-world tasks.

The evaluation covers six simulation tasks and six real-world tasks across two robotic platforms, demonstrating the applicability of the proposed framework to different contact-rich manipulation scenarios.

The reported improvements over tactile-conditioned and tactile-reactive baselines are encouraging. In particular, the ablation studies show that execution-time tactile updates and EATA both contribute to improved performance, providing empirical support for the overall approach.

Critical weaknesses
  1. Limited methodological novelty beyond existing streaming generation frameworks.

The main framework builds heavily upon existing ideas in streaming diffusion and flow-based action generation, where future actions are progressively refined while intermediate generation states are retained.

Similar concepts have already been explored in prior works such as Diffusion Forcing and FASTER. Therefore, progressive action generation and refinement should not be considered major methodological contributions of this paper.

The most distinctive component is Execution-Aware Tactile Attention (EATA). However, EATA is implemented as a relatively straightforward attention mask that restricts direct tactile conditioning to the next action block. While the motivation is reasonable, the technical novelty of this design appears somewhat limited.

More importantly, the paper does not sufficiently establish whether the observed improvements arise specifically from execution-aware temporal alignment or from a simpler attention sparsity or regularization effect. Comparisons with alternative temporal attention strategies would be necessary to better substantiate the claimed contribution.

  1. Insufficient analysis of the streaming schedule and real-time responsiveness.

The proposed framework relies on a predefined block-wise flow schedule, but the interaction between this schedule and execution-time tactile feedback is not sufficiently investigated.

According to Eq. (5), when the k-th block becomes the next block to execute, its flow time has already reached (k-1)/k. For example, with eight blocks, the final block has completed 87.5% of its flow integration before receiving direct access to the tactile feedback aligned with its execution stage.

This raises an important question about how effectively newly acquired tactile information can modify actions that are already close to completion, particularly when unexpected contact events occur.

Furthermore, the paper does not provide sufficient measurements of inference latency, tactile feedback latency, or closed-loop response time. Such measurements are important for demonstrating the practical advantages of streaming generation in real-time contact-rich manipulation.

The evaluation also lacks a computationally matched comparison against straightforward short-horizon replanning with updated tactile observations. Consequently, it remains unclear whether retaining intermediate flow states provides a substantial advantage over simpler alternatives.

Questions for authors
  1. Novelty and comparison with existing streaming methods.

The block-wise streaming generation mechanism appears closely related to existing streaming diffusion and flow-based policies, particularly FASTER.

Could the authors clarify the methodological differences beyond incorporating tactile observations and applying an attention mask? What is the fundamental advantage of the proposed framework over directly extending an existing streaming action policy with periodically updated tactile conditioning?

  1. Necessity of EATA and alternative attention strategies.

The improvement from EATA is interesting, but it is unclear whether the gain specifically comes from execution-time alignment or simply from restricting attention.

Could the authors compare EATA with alternative designs, such as allowing the current tactile observation to condition the next two blocks, applying distance-dependent attention weights, or using other temporally structured attention masks?

Additionally, since EATA only restricts direct action-to-tactile attention, could the authors clarify whether outdated tactile information may still propagate indirectly through interactions between action tokens?

  1. Effectiveness of late-stage tactile refinement.

According to the proposed flow schedule, later action blocks have already completed most of their generation process when they receive the tactile feedback corresponding to their execution stage.

For example, the eighth block has reached a flow time of 7/8 before receiving its latest execution-aligned tactile observation.

How much can such a nearly completed action block still be modified by newly acquired tactile feedback?

Could the authors provide quantitative evidence of responsiveness to unexpected contact changes, or investigate whether alternative flow schedules that preserve more refinement capacity improve performance?

  1. Real-time performance and comparison with replanning.

Could the authors report the actual inference latency, effective action execution frequency, tactile update frequency, and end-to-end feedback-to-action latency on the real robotic systems?

It would also be valuable to compare TacForcing against a conventional flow-based policy that performs more frequent replanning using updated tactile observations, while keeping the backbone, training data, and computational budget comparable.

Such a comparison would help establish whether streaming refinement provides a meaningful advantage over simpler closed-loop replanning strategies.

  1. Statistical reliability of the real-world results.

The real-world experiments use only 16 trials per task, meaning that a difference of approximately six percentage points may correspond to a single successful trial.

Could the authors provide confidence intervals or additional repeated evaluations to better establish the statistical reliability of the reported improvements, particularly for tasks where the differences between methods are relatively small?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PpiN (1/3): Contribution and EATA

Thank you for your careful review and constructive questions. Our responses are organized into three parts.

Q1 & W1. Contribution and distinction from existing streaming policies

“Similar concepts have already been explored in prior works such as Diffusion Forcing and FASTER. Therefore, progressive action generation and refinement should not be considered major methodological contributions of this paper.”

“Could the authors clarify the methodological differences beyond incorporating tactile observations and applying an attention mask? What is the fundamental advantage of the proposed framework over directly extending an existing streaming action policy with periodically updated tactile conditioning?”

As acknowledged in our Method and Related Work sections, TacForcing builds on established streaming-generation ideas. Our contribution is to formulate streaming generation for tactile control, enabling rapidly changing contact feedback to revise an ongoing action chunk without discarding its intermediate states. This addresses the tension between retaining the shared context of chunked actions and adapting to contact changes during execution.

Within this formulation, periodically refreshing tactile inputs addresses observation freshness, but does not by itself align their use with the different execution times of unfinished actions. A tactile observation describes the current contact state, which may be relevant to the next action block but no longer appropriate when later blocks execute.

TacForcing therefore advances direct tactile conditioning with execution progress, while retaining joint refinement of the unfinished chunk. The same correspondence is incorporated into training. Its advantage over unrestricted periodic conditioning is to account for both when feedback becomes available and which actions it should directly guide, reducing the temporal mismatch between tactile observations and action execution.

Q2. Source of EATA’s gains and alternative attention designs

“The improvement from EATA is interesting, but it is unclear whether the gain specifically comes from execution-time alignment or simply from restricting attention.”

“Could the authors compare EATA with alternative designs, such as allowing the current tactile observation to condition the next two blocks…”

We have added these comparisons to Section 4.4, Table 4. Both variants retain sparse direct tactile attention but alter EATA’s execution-aligned conditioning. Delayed feedback preserves the identical mask but uses tactile observations from two blocks earlier. Two-block conditioning extends direct access to the latest observation from the next executable block to the following block as well. Both variants are retrained with their respective rules applied during training and inference, keeping all other settings unchanged.

MethodPull-out Key (%)Lift Can (%)Insert Hole (%)Close CapInsert BatteryClose Lid
Delayed feedback3852534/168/162/16
Two-block conditioning4161626/1610/164/16
TacForcing (EATA)4863699/1613/168/16

Simulation: success rates over 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.

EATA achieves higher observed success rates on all six tasks. In particular, the degradation with delayed feedback under an unchanged mask supports temporal alignment between tactile feedback and action execution as an important source of EATA’s gains, beyond attention restriction alone. Indirect propagation through action tokens remains, but its relative influence on future blocks is limited in our single-forward-pass analysis (Appendix F).

AuthorsOpenReview ↗

Response to Reviewer PpiN (2/3): Latency and Replanning

Q4 & W2. Real-world latency and comparison with replanning

(a) Inference latency and feedback frequency

“Could the authors report the actual inference latency, effective action execution frequency, tactile update frequency, and end-to-end feedback-to-action latency on the real robotic systems?”

We have added these measurements to Section 4.4, Table 3, with component timings and tactile-to-command latency in Appendix A.3. Measurements cover 96 blocks after warm-up on Sharpa North using an RTX 4090 (24 GB), with B = 5, S = 2, and 60 Hz action execution within blocks. Inference and execution do not overlap, and no additional inference acceleration is applied.

MetricResult
First-block latency (VLM + tactile + S steps)60.0 ms
Subsequent-block latency (tactile + S steps)14.8 ms
Effective action rate48.3 Hz
Tactile update rate9.66 Hz
Tactile update rate (excluding action execution)50.3 Hz
Tactile-to-command latency36.4 ms

Mean latencies include preprocessing, computation, and command dispatch. “First block” refers to each chunk’s first block, “tactile” denotes tactile encoding, and S = 2 flow-matching steps. The 48.3/9.66 Hz rates include execution and inference pauses. Removing the measured 80.8% execution-time share gives 9.66/(1 − 0.808) ≈ 50.3 Hz, a processing-only rate. The 36.4 ms spans decoded tactile arrival at the host buffer to the first conditioned command being sent, excluding sensor transmission and actuator response.

(b) Comparison with frequent replanning

“It would also be valuable to compare TacForcing against a conventional flow-based policy that performs more frequent replanning using updated tactile observations, while keeping the backbone, training data, and computational budget comparable.”

We have added this comparison to Section 4.4, Table 4. We separately train a conventional flow-matching baseline with the same backbone, data, tactile encoder initialization, and training hyperparameters. It replans from noise every B = 5 actions using fresh tactile feedback and matched sampling steps per update (S = 5 in simulation, S = 2 on Sharpa North).

The variant with cached VLM features has inference and deployment latencies comparable to TacForcing. Recomputing VLM features at every update adds approximately 215 ms of GPU computation per 40 actions on Sharpa North (7 × 30.77 ms).

MethodPull-out Key (%)Lift Can (%)Insert Hole (%)Close CapInsert BatteryClose Lid
Replanning1643435/169/164/16
Replanning + cached VLM2039367/166/165/16
TacForcing4863699/1613/168/16

Simulation: 100 trials per task. Real world: successes out of 16 trials per task on Sharpa North.

TacForcing achieves higher observed success rates on all six tasks, including against cached replanning with comparable latency. Repeated restarts may disrupt temporal consistency, whereas TacForcing incorporates fresh tactile feedback by retaining and refining unfinished blocks within a continuing action chunk.

AuthorsOpenReview ↗

Response to Reviewer PpiN (3/3): Statistical Reliability

Q5. Statistical reliability of the real-world results

“Could the authors provide confidence intervals or additional repeated evaluations to better establish the statistical reliability of the reported improvements, particularly for tasks where the differences between methods are relatively small?”

Thank you for raising this concern. We conducted 16 additional trials for every method on each of the six real-world tasks, increasing the evaluation to 32 trials per method per task. The main results table now reports successful trials out of 32, making differences in success counts directly visible. We have also added task-specific success criteria and initial object-position settings to Appendix B.2.

The corresponding success rates and 95% Wilson confidence intervals are reported below.

MethodClose CapInsert BatteryClose LidTransfer LiquidWipe BoardStand BottleAvg.
π₀.₅18.8 [8.9, 35.3]40.6 [25.5, 57.7]12.5 [5.0, 28.1]6.3 [1.7, 20.1]37.5 [22.9, 54.7]50.0 [33.6, 66.4]27.6
GR00T N1.734.4 [20.4, 51.7]65.6 [48.3, 79.6]28.1 [15.6, 45.4]12.5 [5.0, 28.1]53.1 [36.4, 69.1]65.6 [48.3, 79.6]43.2
FTP-143.8 [28.2, 60.7]65.6 [48.3, 79.6]31.3 [18.0, 48.6]9.4 [3.2, 24.2]68.8 [51.4, 82.0]75.0 [57.9, 86.7]49.0
T-Rex53.1 [36.4, 69.1]46.9 [30.9, 63.6]46.9 [30.9, 63.6]28.1 [15.6, 45.4]62.5 [45.3, 77.1]78.1 [61.2, 89.0]52.6
TacForcing62.5 [45.3, 77.1]75.0 [57.9, 86.7]56.3 [39.3, 71.8]43.8 [28.2, 60.7]78.1 [61.2, 89.0]81.3 [64.7, 91.1]66.1

Entries show success rates (%) [95% Wilson CI] over 32 trials per method per task. Avg. is the mean across six tasks. Bold indicates the highest observed rate, without implying statistical significance.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Revisions

Dear Reviewer PpiN,

As the discussion period draws to a close, we would appreciate your feedback on our rebuttal and revised manuscript. Following your helpful comments, we have included:

  • Clarification of TacForcing’s contribution and comparisons with alternative tactile-conditioning designs.
  • Hardware latency measurements and frequent-replanning comparisons, including a cached-VLM variant with comparable latency.
  • An expanded main real-world evaluation with 32 trials per method per task and confidence intervals.

If these additions help address your concerns, we would be grateful if you could reconsider your assessment and rating. We also welcome any remaining questions and would be happy to clarify further.

Thank you again for your time and consideration.

Reviewer_YyviOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel way to integrate tactile sensing into visuo-motor policies. A standard approach trains an action expert that takes vision and other sensor modalities (proprioception, tactile, etc.) as input and outputs an action chunk that is executed on a robot.

The authors propose TacForcing, which is a streaming action generation framework. The key idea is that tactile signals can evolve faster than vision, and hence later action steps should be conditioned on up-to-date tactile information. The authors use flow matching with block-wise flow scheduling. Additionally, they use Execution-Aware Tactile Attention (EATA), which prevents later blocks from conditioning on stale tactile information.

The authors show results using Sharpa hand on dexterous tasks both in simulation and in real life, outperforming baseline models with and without tactile.

Critical strengths
  • Real-world results
  • Real dexterous tasks instead of simple toy tasks
  • Sound idea of streaming action expert
Critical weaknesses

None

Questions for authors
  • How much data was used for training in the real world? Was it all clean success data, or was there any DAgger-style data collection?
  • Line 256: "thereby conditioning later blocks on tactile feedback that may become outdated before their execution". What if, instead of masking, we do condition but update it to the latest value in the subsequent denoising step?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Yyvi

Thank you for your positive assessment and helpful questions. We clarify the training data and tactile conditioning below.

Q1. Real-world training data and collection

“How much data was used for training in the real world? Was it all clean success data, or was there any DAgger-style data collection?”

As reported in Appendix A.2, we use 100 demonstration trajectories per real-world task, except for Close Cap and Close Lid, which use 150 and 300 trajectories, respectively. This totals 850 trajectories across six tasks.

All demonstrations were collected offline through human teleoperation, without DAgger-style data collection or aggregation. Although every demonstration ends in successful task completion, we retain trajectories with imperfect intermediate motions or repeated attempts. For example, the Close Lid demonstrations include cases where closing the tea-box lid required several attempts.

Q2. Updating tactile conditioning without masking

“What if, instead of masking, we do condition but update it to the latest value in the subsequent denoising step?”

We evaluate this alternative as variant (ii), “tactile updates without EATA,” in Figure 5. In our implementation, tactile conditioning is refreshed after each executed block. Without EATA, all unfinished blocks attend to the latest observation during subsequent denoising steps.

Adding EATA improves average success from 51% to 60% across the three simulation tasks and from 48% to 69% across the three real-world tasks, with gains on all six tasks. These findings suggest that updating tactile observations alone does not resolve the temporal mismatch between tactile feedback and action execution. EATA helps mitigate this mismatch by aligning direct tactile conditioning with the actions about to be executed.

Reviewer_YyviOpenReview ↗

Thank you

Thank you for answering my questions

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.