- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Exec-VTLA, a manipulation policy that conditions action generation on recent measured robot-state transitions and tactile observations, in addition to visual-language context and current proprioception. A query-based action decoder jointly attends to these information sources and predicts an action chunk relative to a common reference state. The method is evaluated on six UniVTAC tasks and four physical-robot tasks, achieving average success rates of 68.17% and 80.00%, respectively. Component ablations examine tactile input, measured motion history, fusion design, and action chunking.
Critical strengths
- The controlled comparisons support the practical value of conditioning on both motion and tactile signals rather than relying only on current observations.
- The physical experiments further demonstrate that the complete sensing and control pipeline is deployable, which strengthens the practical relevance of the approach.
Critical weaknesses
- The overall advantage is small and strongly task-dependent, and the largest regression remains insufficiently explained. Exec-VTLA improves Put Bottle success from 47% to 80% relative to FTP-1, but achieves only 62% on Lift Bottle versus 97% for FTP-1, resulting in an average improvement of 1.50 percentage points (Table 1). The explanation that lifting relies more on existing policy priors is plausible, but does not identify the source of this substantial performance gap. Indeed, the internal ablations improve Lift Bottle performance when tactile input and measured motion history are included (Table 2), suggesting that the gap cannot simply be attributed to these feedback signals. A matched baseline and task-stage failure analysis would help explain both the Lift Bottle deficit and the extent to which execution feedback accounts for the Put Bottle gain.
- Removing measured motion history reduces average success from 68.17% to 62.17% (Table 2), supporting the usefulness of historical information. However, this comparison does not distinguish the benefit of measured state transitions from that of additional temporal context. Comparisons with commanded-action history and a corresponding sequence of past measured states would help establish the specific advantage of the proposed representation. Likewise, the improvement over concatenation-based fusion, from 64.17% to 68.17% (Table 2), supports the proposed fusion design but does not separately establish the claimed alignment between execution outcomes and future action positions.
- Removing tactile input reduces average success from 68.17% to 58.83% (Table 2). However, the correlation between tactile changes and joint-motion magnitude in one successful rollout (Figure 4, Section 4.5) does not establish that tactile feedback directly causes corrective actions: motion itself can change tactile observations, and joint-motion magnitude does not distinguish correction from continued execution. Furthermore, Algorithm 1 and Appendix C.3 specify that all eight predicted action rows are executed without regeneration within the current chunk. Updated feedback therefore conditions the next chunk rather than revising the remaining actions in the current one. This still constitutes feedback control across chunks, but without measurements of the policy-update interval or controlled unexpected-contact tests, the claimed responsiveness remains insufficiently characterized.
Questions for authors
- How does measured-transition history compare with commanded-action history and past-state history under identical architecture, history length, and training budget?
- Were Exec-VTLA and FTP-1 evaluated on identical initial conditions? Can you provide paired statistics or repeated-run uncertainty for their aggregate difference and explain the substantial Lift Bottle regression?
- What is the actual time between policy updates when all eight action rows are executed? Can unexpected contact trigger interruption or regeneration of the current chunk?
- How are common-anchor action predictions converted into commands during sequential execution? In particular, is each row reconstructed relative to the original anchor or applied relative to the latest measured state?
- How is the scalar contact estimate used in Figure 4 constructed, and is it used by the policy itself? Can a controlled change in contact or execution outcome demonstrate that the corresponding feedback changes the subsequent correction, beyond the correlation in one successful rollout?
8 comments
## Response to Reviewer 8SL6 (Part 2 / 5)
A1 (continued).
Put Bottle Analysis. The submitted Table 2 directly evaluates the contribution of execution feedback on Put Bottle: the full model reaches 80%, compared with 71% without tactile input, 74% without measured motion history, 76% with concat fusion, and 60% with the H1 policy. The corresponding differences are 9, 6, 4, and 20 percentage points. Sec. 4.3 identifies Put Bottle as a task in which the next action depends on the displacement produced by the preceding action, and Fig. 4 in Sec. 4.5 illustrates the contact, adjustment, and convergence stages of this same task. These task-specific results connect the success-rate improvement to the execution signals and decoder configuration.
Q2: Removing measured motion history reduces average success from 68.17% to 62.17% (Table 2), supporting the usefulness of historical information. However, this comparison does not distinguish the benefit of measured state transitions from that of additional temporal context. Comparisons with commanded-action history and a corresponding sequence of past measured states would help establish the specific advantage of the proposed representation. Likewise, the improvement over concatenation-based fusion, from 64.17% to 68.17% (Table 2), supports the proposed fusion design but does not separately establish the claimed alignment between execution outcomes and future action positions.
A2: R1-Table3 reports the requested history-source controls. The network, history length, and training schedule are fixed; only the history content changes among raw past states, issued commands, and measured transitions.
R1-Table3: History-source and component ablations on UniVTAC. Success rates (%). The three history-source variants use the same architecture, history window , and training budget. The raw-state and commanded-action rows are additional controls; the remaining rows reproduce Table 2 of the submitted paper. Bold indicates the highest success rate in each column.
| Experiment setting | LiftBottle | PullKey | LiftCan | PutBottle | InsHole | InsTube | Avg. |
|---|---|---|---|---|---|---|---|
| w/o tactile | 54 | 38 | 58 | 71 | 59 | 73 | 58.83 |
| w/o measured action history | 57 | 42 | 62 | 74 | 62 | 76 | 62.17 |
| concat fusion | 59 | 44 | 64 | 76 | 64 | 78 | 64.17 |
| (raw past-state history) | 58 | 52 | 63 | 66 | 63 | 80 | 63.67 |
| (commanded-action history) | 58 | 51 | 64 | 74 | 65 | 79 | 65.17 |
| H1 single-step policy | 48 | 28 | 51 | 60 | 50 | 57 | 49.00 |
| Full Exec-VTLA () | 62 | 50 | 68 | 80 | 67 | 82 | 68.17 |
Results Analysis. The measured-transition variant achieves an average success rate of 68.17%, compared with 63.67% for raw-state history and 65.17% for commanded-action history. It improves success rates on five of six tasks in both comparisons. On Pull-out Key, it scores 2 percentage points below raw-state history and 1 percentage point below commanded-action history. The first comparison evaluates the benefit of explicitly representing realized displacement; the second compares realized motion with requested motion.
Implementation. Each future action row has its own learned query, and every query attends to the shared execution histories through Eqs. (6)–(7). The Exec Block applies standard softmax attention to concatenated keys and values, using separate projections for each branch and accessing the conditions at every decoder block. The concat-fusion baseline combines the conditions at the input. The full model improves success rates on all six tasks by 3–6 percentage points relative to this baseline, supporting the complete fusion design.
## Response to Reviewer 8SL6 (Part 3 / 5)
Q3: Removing tactile input reduces average success from 68.17% to 58.83% (Table 2). However, the correlation between tactile changes and joint-motion magnitude in one successful rollout (Figure 4, Section 4.5) does not establish that tactile feedback directly causes corrective actions: motion itself can change tactile observations, and joint-motion magnitude does not distinguish correction from continued execution. Furthermore, Algorithm 1 and Appendix C.3 specify that all eight predicted action rows are executed without regeneration within the current chunk. Updated feedback therefore conditions the next chunk rather than revising the remaining actions in the current one. This still constitutes feedback control across chunks, but without measurements of the policy-update interval or controlled unexpected-contact tests, the claimed responsiveness remains insufficiently characterized.
A3: In Sec. 4.5, “correction” refers to contact-dependent adjustment of subsequent actions within a rollout. The model learns this conditional action mapping through behavior cloning: measured transitions and tactile tokens augmented with the projected contact prediction enter the Exec Block and condition each new action chunk.
Execution Analysis. Fig. 4 follows the approach contact/adjustment convergence process. Before contact, the contact estimate remains near its baseline. During active adjustment, tactile variation and joint motion increase together ( between and in the illustrated rollout). As execution settles, the maximum joint jump decays from 0.038 to 0.0016. This phase-resolved correlation provides mechanism-consistent evidence of a contact–adjustment–convergence chain within successful execution.
The multi-rollout convergence analysis gives a tail-to-peak joint-jump ratio of for successful rollouts and for failed rollouts. The ablations and stage-wise results provide complementary evidence that tactile and measured-motion feedback reduce failures during lift initiation and after lifting. Together with the contact-prediction path described in A8, these results support our interpretation of correction. The timing and interruption mechanisms are detailed in A6.
Q4: How does measured-transition history compare with commanded-action history and past-state history under identical architecture, history length, and training budget?
A4: R1-Table3 in Part 2 is the requested controlled comparison. Given the current state and the same state window, raw-state and measured-transition histories are information-equivalent: transitions can be recovered by differencing consecutive states. The advantage investigated here is therefore representational: measured transitions make each realized displacement directly available to the action decoder.
Commanded-action history records the requested motion, whereas measured transitions record the realized motion. In R1-Table3, measured transitions achieve the highest average success rate and improve five of six tasks relative to each alternative. Under the same architecture, history length, and training budget, these results support representing realized motion explicitly as state transitions.
## Response to Reviewer 8SL6 (Part 1 / 5)
We thank Reviewer 8SL6 for the careful assessment and for recognizing the value of the controlled comparisons and the deployability of our sensing and control pipeline. Your comments help us clarify the task-dependent results, the contribution of execution history, and the timing of feedback. We address each concern and question below.
Q1: The overall advantage is small and strongly task-dependent, and the largest regression remains insufficiently explained. Exec-VTLA improves Put Bottle success from 47% to 80% relative to FTP-1, but achieves only 62% on Lift Bottle versus 97% for FTP-1, resulting in an average improvement of 1.50 percentage points (Table 1). The explanation that lifting relies more on existing policy priors is plausible, but does not identify the source of this substantial performance gap. Indeed, the internal ablations improve Lift Bottle performance when tactile input and measured motion history are included (Table 2), suggesting that the gap cannot simply be attributed to these feedback signals. A matched baseline and task-stage failure analysis would help explain both the Lift Bottle deficit and the extent to which execution feedback accounts for the Put Bottle gain.
A1: Thank you for highlighting the task-level differences. As reported in Table 1, Exec-VTLA achieves the highest average success rate and the best results on four of the six tasks: Pull-out Key, Put Bottle, Insert Hole, and Insert Tube. We reproduce the comparison below and then examine the Lift Bottle failures.
R1-Table1: Comparison with SOTA methods on UniVTAC, reproduced from Table 1 of the submitted paper. Success rates (%). FTP-1 values are its published results. Methods evaluated under our protocol use 100 seeds per task; Avg. is the unweighted mean across six tasks.
Lift Bottle remains the main limitation: FTP-1 achieves 97%, compared with 62% for Exec-VTLA. FTP-1 benefits from large-scale tactile-policy pretraining, while our policy training uses 100 demonstrations per UniVTAC task (Appendix C.1). Exec-VTLA uses pretrained vision-language and tactile representations but does not undergo comparable large-scale tactile-policy pretraining. This difference offers an explanation for the gap on a task that relies strongly on grasping and lifting priors, as discussed in Sec. 4.2. The stage-wise analysis below identifies where the failures occur.
Failure Analysis. To identify the stages associated with the performance gap, we classify failures into three mutually exclusive categories: failure to lift (A), post-lift drop (B), and placement/upright failure (C). R1-Table2 reports the percentage of all 100 evaluation episodes assigned to each outcome.
R1-Table2: Stage-wise failure analysis on Lift Bottle. Percentages of all 100 evaluation episodes for each variant.
Results Analysis. The gain of 8 percentage points over the variant without tactile input corresponds to 7 fewer failures to lift and 1 fewer post-lift drop, with placement/upright failures unchanged. Similarly, the gain of 5 percentage points over the variant without measured history corresponds to 4 fewer failures to lift and 1 fewer post-lift drop. Tactile and measured-motion feedback therefore improve lift initiation and, to a smaller extent, post-lift stability.
FTP-1 fails in 3 of 100 episodes, all ending when the action budget is exhausted with the bottle still tilted. Exec-VTLA's failures occur mainly before placement: 15 during lift initiation and 12 after the bottle has been lifted. The ablations show that execution feedback improves these stages within Exec-VTLA, while the comparison with FTP-1 identifies lifting and post-lift stability as the main remaining weaknesses. We will include this analysis in Sec. 4.2.