- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RoboTaskonomy studies which similarities between robot skills predict transfer. It compares task attributes, complete trajectories and local motion distributions against measured adaptation outcomes, then factorizes the motion distributions into six shared components using external AgiBot data. These component mixtures define skill kinship and select prior adapters to initialize new skills. The paper evaluates transfer rankings across four policy families, composition stability on held-out tasks, and continual learning in simulation and on a dual-arm robot.
Critical strengths
I like that the representation is checked against actual policy transfer: across 144 directed pairs, local motion reaches a mean within-target correlation of 0.376, with a 95% interval of [0.185, 0.551] (Table 8). The quantizers and basis are fitted on external AgiBot data without transfer outcomes, which makes the evidence more convincing.
The paper connects this analysis to a practical intervention: ReL raises final simulation success from 16.4% to 26.0% against a random-source baseline using the same adapter framework (Table 14). The held-out composition checks and real-robot experiment add useful support.
Critical weaknesses
I'm not convinced that the discovered primitives improve source selection beyond the original motion distributions. In Table 12, composition alone has a lower point estimate than raw distributions (0.372 versus 0.429); the best result uses fusion, while ReL uses composition alone and lacks a raw-motion retrieval control.
The continual-learning comparison uses one fixed task order in each setting and one sampled Random source history, with no repeated-run uncertainty reported (App. E). On the robot, 32% versus 24% is only four additional successes out of 50 final trials per method, so the reliability of that gain is still unclear.
Questions for authors
How does ReL compare with raw-motion similarity used for source selection in the same adapter framework? If they perform similarly, I'd view the NMF basis mainly as an interpretability tool.
Does the simulation advantage persist across task orders, training seeds and independent Random source draws? Repeated paired runs would materially strengthen my confidence in the practical result.
2 comments
W2 & Q2: Experiments on randomness and different task orders.
Thank you for this suggestion. We provide detialed results on different task orders, random source selection, and random seeds.
1. Different task order.
We evaluated two additional orders of the same ten tasks, keeping the training budget and evaluation protocol fixed across methods. ReL outperforms Random in both additional orders, by 12.8% and 9.6%, complementing the 9.6% gain reported for the original order. The advantage over random-source initialization persists across all three evaluated orders, which support the robustness of relation-guided source selection beyond the original task sequence.
| Task order | ReL Final (%) | Random Final (%) | CLARE Final (%) | SeqFT Final (%) | ReL − Random (pp) |
|---|---|---|---|---|---|
| Original order (submitted paper) | 26.0 | 16.4 | 17.2 | 11.6 | +9.6 |
| Additional order 1 | 20.8 | 8.0 | 15.2 | 9.6 | +12.8 |
| Additional order 2 | 24.4 | 14.8 | 15.2 | 14.8 | +9.6 |
2. Different random sequence for the baseline.
To assess the robustness of the Random baseline to source-selection history, we supplement the original history (H1) with two additional histories (H2 and H3), all fixed before training, while keeping the task order and optimization seed unchanged. Random source selection yields limited and inconsistent gains across these inheritance histories, with improvements on some tasks offset by declines on others. These results highlight the importance of characterizing transfer relationships to guide prior-skill selection.
| Random selection history | Reproduction seed | Final (%) | Mean (%) |
|---|---|---|---|
| H1 (original) | 444298 | 16.40 | 7.57 |
| H2 (additional) | 147494 | 18.80 | 9.24 |
| H3 (additional) | 228465 | 17.20 | 8.68 |
| Average | — | 17.47 1.22 | 8.50 0.85 |
3. Repeated Run.
The original experiments used a training seed of 42. We additionally train and evaluate ReL and Random using seeds 43 and 44, and report the mean and standard deviation across these additional runs. ReL outperforms Random with both seeds, yielding an average improvement of 10.93% in the Final score. These results provide additional evidence that the benefit of relation-guided initialization persists across different repetitions.
| Training seed | ReL — Mean (%) | ReL — Final (%) | Random — Mean (%) | Random — Final (%) |
|---|---|---|---|---|
| 42 | 13.01 | 26.00 | 7.57 | 16.40 |
| 43 | 15.68 | 27.60 | 7.32 | 14.40 |
| 44 | 14.26 | 24.40 | 9.80 | 14.40 |
| Average | 14.32 1.34 | 26.00 1.60 | 8.23 1.37 | 15.07 1.15 |
4. Robustness on real deployment.
To further evaluate the robustness of our results on the physical robot, we conducted four additional runs, bringing the total number of runs to five. The results are summarized below, with values reported as mean ± sample standard deviation across the five runs. ReL achieves an average improvement of 8.0% over Random. These repeated evaluations provide further evidence that the practical benefits of relation-guided skill reuse are consistent across repeated physical-robot trials.
| Method | Final (%) |
|---|---|
| CLARE | 16.00 ± 2.00 |
| Random | 23.20 ± 2.28 |
| ReL | 31.20 ± 3.35 |
W1 & Q1: Significance of primitives beyond raw distribution.
First, we agree with the reviewer that primitive compositions and raw-motion distributions achieve comparable effectiveness in source selection. However, our primary motivation for extracting primitives is to investigate the shared structure underlying the transfer-relevant representation. The discovered primitives reveal recurring behavioral constituents, make overlapping relationships among skills explicit, and provide a consistent basis for characterizing unseen skills. Their scientific value lies in uncovering how diverse skills are organized through shared components and distinct compositions, with downstream source selection serving as one application of this structure.
Second, we further evaluate raw-motion- and fusion-guided source selection on the original ten-task sequence. Raw-motion similarity and primitive composition select the same source in 7 of 9 decisions. The resulting final success rates show that primitive composition preserves effective source-selection information, while fusion provides an additional empirical improvement.
The transfer analyses provide complementary evidence. Across the 24 target tasks, composition-selected sources yield a mean transfer gain of 14.25%, compared with 13.58% for raw motion. Taken together, these results support composition as an effective source-selection criterion that also exposes the shared behavioral structure underlying skill relationships.