- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces TS-CFG, a method that constructs DPO preference pairs for flow-matching VLAs using only existing demonstration data. It first inverts demonstration actions into latent variables within the original context and then reconstructs them using a frozen policy; the velocity field blends predictions derived from the original context and the context of adjacent frames within the same episode, with the weight of the temporal context decaying from 1 to 0 during reconstruction. This process yields hard negative samples, against which the demonstration actions are prioritized using a Diffusion-DPO-style loss. Improvements were achieved on the π0.5 model across the RoboTwin (Easy/Hard) and RoboCasa365 benchmarks.
Critical strengths
Strengths:
1.This is a simple and practical method. It requires neither new rollout or manual annotation stages nor reward or world models, and the deployment-time sampler remains unchanged. "Demonstration inversion combined with temporal context guidance" is a novel approach with a clear rationale, where the generated hard negative samples are tied to specific demonstrations.
2.The setup for the simulation experiment assignments is meticulous and reliable.
Critical weaknesses
Weakness:
1.The evidence supporting the main reported results is relatively weak; all results reflect values from a single random seed, and the improvement over in the primary experiments is marginal. Furthermore, the simulation experiments only evaluated , and real-world robot experiments are missing.
2.There is no evidence that negative samples of action generated via perturbation are inferior, nor is there a description of how the perturbation alters the original demonstration.
Questions for authors
- Please provide the mean and standard deviation (std) results across multiple random seeds.
- What happens when "aaaa" is introduced into the environment, and what is the success rate?
- Can the TS-CFG method be generalized to other VLA models or real-world robots?
2 comments
Response to Reviewer JVoc (Part 2/2)
W2 / Q2. Which behavioral changes follow preference training?
The clearest change is more reliable completion after the robot has already made useful progress. On RoboTwin Hard, TS-CFG improves over base on 35 of 50 tasks, using its three-seed mean. Grouping all 50 tasks by their manipulation requirements gives the following descriptive breakdown; each task appears once and rates are averaged equally across tasks:
| Task group | Tasks improved / total, vs. base | Base | TS-CFG |
|---|---|---|---|
| Articulated opening and switching | 2/3 | 46.00 | 62.22 |
| Button pressing and direct gripper contact | 3/3 | 68.67 | 79.33 |
| Hand-to-hand object transfer | 2/2 | 35.00 | 45.33 |
| Multi-object placement, ordering and stacking | 8/10 | 55.60 | 63.47 |
| Container manipulation sequences | 4/5 | 51.20 | 56.67 |
| Lifting and held-object manipulation | 8/9 | 67.33 | 70.74 |
| Single-object relocation and pose adjustment | 8/16 | 59.38 | 62.71 |
| Held-tool targeting | 0/2 | 68.00 | 63.00 |
Seven of eight group averages improve, with gains in both handover tasks, all three direct-contact tasks and eight of ten multi-object tasks. This points to better execution through changing contact and task stages, rather than an improvement confined to one object or task family.
Representative full-evaluation results are:
| Task | Base (%) | TS-CFG, three-seed mean (%) |
|---|---|---|
| Cup placement | 40.00 | 84.67 |
| Microwave opening | 30.00 | 74.00 |
| Two-can placement | 38.00 | 68.67 |
| Microphone handover | 42.00 | 58.00 |
The saved rollouts show where these differences arise. We inspected a fixed, outcome-independent sample of ten scenes per task and policy, using the original TS-CFG training seed:
- Cup placement: both policies reach the coaster region in 10/10 scenes. Base completes placement in 4/10, versus 10/10 for TS-CFG. The separation occurs after the cup has reached the correct target region, not in identifying where to move it.
- Two-can placement: at least one can reaches the box in 8/10 base scenes and 10/10 TS-CFG scenes, but both are placed in 1/10 versus 6/10. The larger difference is in progressing from a partially completed task to the second successful placement.
- Microwave opening: base can initiate door movement but fail to complete the opening; TS-CFG rollouts continue the interaction to the successful open state. This is a difference in completing the contact-dependent motion, not simply reaching the appliance.
An additional paired microphone-handover example illustrates object retention. Both policies grasp the microphone. Base loses it during the transfer between grippers; TS-CFG retains it and finishes the handover. The relevant distinction is maintaining the object through a change in which hand controls it.
These stages explain why temporally nearby alternatives are useful. The task and objects remain the same across nearby frames, but the action needed before contact, while holding an object, or after reaching its target changes. A policy can therefore produce a task-relevant action that is mistimed for the current hand-object relationship. TS-CFG builds this local mismatch into the reconstruction of the same demonstration and decreases its influence before reconstruction ends. Preference training then favors the demonstrated, current-state action over that plausible alternative. This connects the construction to the observed improvements in sustaining transfer, completing placement and continuing a multi-object sequence, rather than attributing them only to generic task recognition.
W1 / Q3. Other VLA models and real robots
We also evaluated TS-CFG with GR00T N1.6 on RoboCasa full50:
| GR00T N1.6 condition | Success (%) |
|---|---|
| Base | 19.72 |
| TS-CFG | 22.32 |
The 2.60-point improvement extends the evidence to a second flow-policy backbone, without changing its deployment sampler. The method acts through the conditional velocity field, so its construction is not specific to pi0.5's architecture. Together with the diverse bimanual and household tasks in RoboTwin and RoboCasa, this supports applicability across simulated tasks, environments and backbones. Real-robot evaluation is the next extension to test the same construction under physical sensing and execution conditions.
Response to Reviewer JVoc (Part 1/2)
Thank you for recognizing the demonstration-specific construction and the value of self-supervised preference generation. We have added repeated runs, direct execution analysis and a second policy backbone.
W1 / Q1. Mean and standard deviation across training seeds
We completed three independent training seeds for every main-table post-training method, using the same settings without retuning. Each policy is evaluated on 50 tasks with 50 rollouts per task. Results are mean +/- sample standard deviation (%):
TS-CFG exceeds the strongest baseline mean by 2.79 / 1.96 / 2.33 percentage points, with improvements across independent training runs on all three evaluations.
The performance gap becomes substantially larger with full-parameter fine-tuning. We used LoRA in the main experiments for computational efficiency. We additionally evaluated all three preference methods with full updates on RoboTwin Hard50, using the same initialization, data, global batch 8 and 5k-update budget. These are single-seed comparisons at seed 31; full-parameter runs share LR 1e-8, while the original LoRA runs use 5e-7.
Neither baseline improves over its own LoRA counterpart in this comparison, whereas TS-CFG gains 4.52 points. Under the matched full-parameter settings, TS-CFG exceeds DITTO* and VTLA* by 8.76 and 7.36 points, respectively. Its advantage therefore extends beyond the parameter-efficient setting and is substantially larger under full updates.
W2 / Q2. How is the demonstration changed, and what happens during execution?
We interpret "aaaa" in Q2 as referring to executing the temporally perturbed actions in the environment. Please let us know if you intended something different; the following addresses that interpretation.
The negative is a modified reconstruction of a particular demonstration, not an independently sampled action. For demonstrated action a+, current context o and nearby context h, we obtain the demonstration's latent under the current context and reconstruct it with temporal guidance:
z⋆=To−1(a+),a−=TDEC(z⋆).
At denoising step k, the field is
vk(x,τ)=vo(x,τ)+λk[vh(x,τ)−vo(x,τ)],λk=1−k/9,k=0,…,9.
Both velocities are evaluated at the same intermediate action state and time. The difference between the temporal and current velocity fields changes the reconstruction toward what the policy would predict under a nearby manipulation state. Its weight decreases to zero, so reconstruction finishes under the current context. Inversion preserves the link to the demonstrated behavior; temporal guidance supplies a structured deviation from it. The original demonstration remains the positive.
This design aims at hard negatives that are close in task and behavior but differ in compatibility with the current state. Under the same DPO objective, it reaches 64.08% on Hard50, compared with 60.56% for zero-guidance/noise reconstruction, 58.16% for random initialization and 58.32% for random same-episode conditioning.
Direct execution of the perturbation. We apply guidance from the context five frames in the past to the base policy during online rollouts, using the same linear 1-to-0 schedule. Both guided and unguided rollouts start from random noise because no demonstrated action is available online. With matched execution settings, success falls from 58.24% to 49.48%, a drop of 8.76 points; success decreases on 37 of 50 tasks.
The perturbed policy still completes 49.48% of rollouts: temporal guidance reduces execution reliability without simply destroying task-directed behavior. This is the intended balance for preference construction: expose errors while remaining sufficiently plausible to provide informative negatives for preference training. During training, inversion additionally ties each alternative to its demonstrated positive.