- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes TRACK, a reference-conditioned reinforcement learning framework that represents manipulation tasks using object keypoint trajectories, grasp flags, and approximate end-effector waypoints. A progress tracker aligns policy execution with the reference. The authors evaluate a single policy on seen and unseen tasks in Meta-World and RoboTwin, including corrupted scripted references and VLM-generated plans.
Critical strengths
- The geometric task representation and state-based progress tracker are technically meaningful.
- The tracker improves robustness to temporal misalignment and generally outperforms clock-based tracking.
- The framework supports single-arm and bimanual manipulation without task-specific success rewards.
Critical weaknesses
The main experimental setting does not provide convincing evidence of generalization to unseen tasks. The reference and test episode are initialized from the same simulator state. Moreover, only the end-effector trajectory is corrupted, whereas the object keypoint trajectory and grasp flags remain exact. The applied corruption is relatively limited, consisting of a smooth displacement of at most 2 cm per axis and Gaussian jitter with a standard deviation of 3 mm. The corruption procedure is also shared between training and evaluation.
Under this protocol, the policy can largely solve the task by reconstructing the exact reference trajectory from a mildly corrupted version. The experiment therefore resembles reference denoising more closely than one-shot execution of multiple unseen tasks. A stronger evaluation should provide the task specification from a different initial state. A reference collected from another robot embodiment would provide more direct evidence for the claimed embodiment independence, but no such experiment is included.
The VLM-plan experiment reinforces this concern. Even on training tasks, one-shot VLM plans achieve only 0.60 success on ML10 and 0.52 on ML45 with the tracker, despite providing an example plan from the same task. These results are substantially below those obtained with scripted references. This suggests that the method depends strongly on having a reference closely aligned with the exact execution instance. Although TRACK outperforms direct reference replay in the aggregate VLM results, the experiment does not establish robust task execution from an independently generated task specification.
On unseen ML45 tasks, TRACK achieves only 25% success with one-shot plans, whereas DCRL[1] reports 48% with one demonstration. Furthermore, DCRL evaluates each episode with independently sampled demonstration and execution initial conditions, providing a stronger test of task-level generalization.
[1] Christopher R. Dance, Julien Perez, and Th ́eo Cachet. Demonstration-conditioned reinforcement learning for few-shot imitation. In International Conference on Machine Learning (ICML), volume 139 of PMLR, pp. 2376–2387, 2021.
Questions for authors
-
Can the authors evaluate TRACK using references collected from initial states different from those of the execution episodes?
-
How does TRACK perform when the corruption type or magnitude differs between training and testing?
-
Can the authors provide a genuine cross-embodiment experiment using references collected from a different robot?
-
How does TRACK compare with DCRL under a common ML45 unseen-task protocol, particularly when the reference and execution initial states are sampled independently?
9 comments
On the VLM results, and 25% vs. DCRL's 48%
The VLM experiment tests references that come from a planner rather than from the simulator, and the two numbers are measured under different protocols.
The plans are made for the episode's own scene, and in a plan no channel is the simulator's record, so they test the case the reviewer raises. There the planner is the bottleneck (on ML10's training tasks, replay scores 0.81 on expert references but 0.40 on one-shot plans of the same scenes). Our planners are open models of at most 27B parameters, far smaller than current state-of-the-art models, and going from an 8B to a 27B planner already improves the plans (App. C.18). Better plans help TRACK more than replay, and with expert references, a stand-in for a near-perfect planner, TRACK reaches 0.95 on its training tasks (App. C.18). The 25% is TRACK on ML45's unseen tasks executed from VLM plans rather than from a demonstration. Conditioned on one demonstration of the same tasks, TRACK reaches 0.74 (Table 1). DCRL's 48% uses Meta-World v1, the observed goal, task rewards and 10 folds over 46 tasks, which leave out two of ML45's test tasks. DCRL's own supplement (Sec. 4.7) reports 45% for a policy that receives no demonstration, against 48% with one. Most of the 48% therefore comes from the observed goal and the task reward, not from the demonstration, and our DCRL-style policy shows the same pattern (App. C.7). For a comparison under one protocol, we trained a DCRL-style policy in our framework (Q4 below, and the table in our reply to R25b, Q1, App. C.7). TRACK is well above it when the reference matches the scene, and, with the reference adapted to the scene, above both DCRL-style arms when demonstration and initial state are drawn randomly and independently.
Q3. References from another robot
We added an experiment in which policies trained on one robot execute demonstrations recorded with another. In short, TRACK executes the other robot's demonstrations and, on average, is above replaying them, on unseen tasks and on the tasks it was trained on.
Setup (App. C.16). A simulated Franka Panda (system-identified controller and gripper) with Meta-World's own objects in its workspace. TRACK's keypoint recipe (K = 4, clock and tracker, 5 seeds each) is trained on the Panda's own demonstrations of ten tasks. A controller turns the wrist to the reference's orientation, and the policy commands the translation and the gripper. At test time, the policies are conditioned on the Sawyer's original Meta-World demonstrations, mapped into the Panda's frame, on the training tasks and on five Meta-World tasks they were never trained on. The two robots differ in hand geometry (the Panda's hand is about 20 cm long).
| Panda, Sawyer demonstrations | train | held-out | one-shot |
|---|---|---|---|
| Replay | 0.56 | 0.56 | 0.48 |
| TRACK (clock) | 0.60 | 0.62 | 0.55 |
| TRACK (tracker) | 0.59 | 0.61 | 0.60 |
TRACK's lead comes from tasks on which replaying the Sawyer's path fails on the Panda. On the training tasks and on window-close, its successes come from following the Sawyer reference, since none remain when the reference is frozen at its first index.
In a second experiment (App. C.17, details in our reply to vYK8, Q1), a simulated KUKA iiwa arm with a five-finger hand is trained and evaluated with the Sawyer's demonstrations as references, which contain no finger motion. TRACK learns the finger motion and is well above replay in distribution and on the five unseen tasks (one-shot 0.65 vs. 0.40 with the tracker). These are two pairs of robots and few unseen tasks, so we report them as first results. For a real robot, including five human demonstrations, see our reply to vYK8, Q4.
Q4. DCRL under a common protocol with random initial states
We trained a DCRL-style policy in our framework, with demonstrations and initial states drawn randomly and independently, as suggested in the review (see our reply to R25b, Q1 and App. C.7). In short, TRACK is well above it when the reference matches the scene. With random initial states, TRACK is at replay's level, but adapting the reference to the scene puts it well above both DCRL-style arms on unseen tasks (0.57 / 0.69 vs. 0.16 / 0.27).
Thank you for the extensive additional experiments. The results for Q2 and Q3 show robustness to inaccurate end-effector paths and to references from another robot. They also clarify what TRACK requires, and this raises my main concern about its intended use.
Generalization
With a reference from another initial state, TRACK drops to 0.34/0.19/0.37 and does not outperform replay. Performance recovers only after the reference is adapted using the current object position and goal. In the main protocol, the object track is the exact object trajectory of the execution episode. In the VLM experiment, the planner receives the goal coordinates. In every working setting, therefore, the object and goal geometry of the current scene is resolved outside TRACK, by the simulator, a planner, or a goal-aware adaptation. TRACK corrects the end-effector path, grasp timing, and embodiment differences of a reference that is already exact for the scene. This makes it a closed-loop executor of scene-specific object plans. It does not show generalization to unseen tasks from an independently obtained demonstration.
the claims and comparisons.
- The absence of task-specific rewards is a property of training. It does not reduce the information TRACK needs at test time. As the rebuttal itself argues for DCRL's 48%, goal information explains much of the performance, and TRACK also receives the goal in every setting. The comparison with DCRL therefore does not establish better task-level generalization.
- If TRACK is an executor, the relevant baselines are other methods that execute the same input, such as trajectory-tracking controllers or policies conditioned on object trajectories. Replay is not sufficient.
- The rebuttal states that the planner is the bottleneck (replay 0.81 with expert references vs. 0.40 with one-shot plans). It also states that moving an object by 5 cm makes most episodes fail. Practical use thus depends on accurate upstream estimation. However, no results are reported with realistic errors in the object and goal positions used to adapt the reference.
I ask the authors to restrict the claims to execution of scene-aligned references, to state explicitly that the goal is available in all settings, and to evaluate adaptation under realistic object and goal estimation errors.
Given the additional evidence on corruption robustness and cross-embodiment transfer, I raise my rating from 1 to 2. The paper remains below acceptance because its central claim of executing tasks from independently obtained references still depends on scene-exact object and goal information.
We thank the reviewer for the careful and critical review. The new evaluations are in purple.
Q1. References from initial states other than the execution's
We added this evaluation, together with a simple adaptation of the reference to the new scene. In short, a reference recorded in another initial state no longer solves the new scene for any method, but the adapted reference brings TRACK back close to its matched performance and above replay.
We evaluate each reference in three settings (ML10, App. C.14). Matched is Table 1's protocol, where the episode starts from the initial state in which the reference was recorded. In another initial state, the same reference runs from the initial state of another demonstration of the same task. Adapted uses that initial state with the reference first adapted to it, by shifting the end-effector path and the object tracks onto the scene's object and moving the end of the carry to the scene's goal. Replay receives the same adapted references, and the goal is given to the adaptation, not to the policy. Matched replay and clock values are those of Table 1:
Every method drops with a reference from another initial state, mostly because that reference no longer fits the scene. Its object sits a median 6.9 cm away, and moving an object by 5 cm already makes every method fail on most episodes. Where the goal is placed independently of the object, the reference also ends at the wrong goal. Adapted to the scene, the reference recovers at least 84% of TRACK's drop, and TRACK is again above replay on every split.
So TRACK tolerates references that are approximate but made for its scene, such as corrupted end-effector paths and grasp flags (Q2), errors in the perceived object position (vYK8, Q4, App. C.13) or demonstrations from another robot (Q3). A demonstration from another scene must first be adapted to it.
Q2. Different corruption type or magnitude in training and testing
We tested the paper's policies at other corruption magnitudes and on corruption types they never saw, and trained policies at other magnitudes. TRACK is above replay under every one of these test corruptions on ML10, at every magnitude from exact references to 4× the training corruption, and under every corruption type we tested, on training, held-out and unseen-task references. The tracker policy is above replay in every case, and the clock policy in every case but one, where it is level (one-shot under a constant offset).
Magnitude (ML10, App. C.15). Cleaner references than in training cost TRACK nothing. At twice the training magnitude it loses about as much as replay, and at three and four times the magnitude its lead over replay is larger than at the training magnitude. Training at the test magnitude, exact or twice as large, helps by at most 0.07, so TRACK does not rely on seeing the test corruption in training.
Types not seen in training. Time warps and grasp flags shifted by 10–20 reference indices cost TRACK at most 0.05, except for the tracker policy with flags 20 indices early, while replay loses up to 0.35 when the flags shift. Replay closes the gripper on the flags, whereas TRACK's grasp timing depends little on them. A constant offset costs TRACK the most overall, at most 0.10, and leaves replay unaffected, because replay subtracts the offset of the first waypoint. Jitter of 1 cm per index, more than three times the 3 mm in training, costs TRACK at most 0.05 and replay up to 0.07. Noisy object tracks, at half the hand's corruption (0.7 cm on average), cost at most 0.01.
On "reference denoising". Three observations argue against the policy simply reconstructing the clean reference. Among the episodes it wins and replay loses on ML10, its path leans toward the uncorrupted track in only 19–41%, while a grasp correction occurs in 66–92% (vYK8, Q1). Corruption types never seen in training cost it little, and training on exact or 2× references helps by at most 0.07 (above).