acceptodds
Under review as a conference paper at ICLR 2027

RoboTaskonomy: Discovering Transfer-Relevant Kinship among Robotic Skills

Abstract

Real-world robots are expected to continually acquire new skills to adapt to changing demands and environments. The ability of prior skills to facilitate new skill acquisition, i.e., skill transferability, is critical for improving learning efficiency. However, skill transferability remains poorly understood, and it is unclear which relationships among skills are relevant to transfer and how they should be characterized. In this paper, we present RoboTaskonomy, a framework for characterizing transfer-relevant relationships among robotic skills and discovering how these relationships are structured. We first construct skill relationships from task attributes, fine-grained trajectories, and local motion distributions, then assess their correspondence with measured transfer effects. This analysis identifies a transfer-relevant representation space in which skill relationships align more closely with measured transfer effects. Building on this representation space, we discover a set of behavioral primitives shared across skills. Skills can be represented through combinations of these primitives, with stable structure across task contexts and unseen skills emerging as novel compositions. We further leverage this structure to guide prior-skill selection in continual learning, significantly improving new-skill acquisition in both simulation and real-world tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation, Skill Transferability, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PMgzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RoboTaskonomy studies which similarities between robot skills predict transfer. It compares task attributes, complete trajectories and local motion distributions against measured adaptation outcomes, then factorizes the motion distributions into six shared components using external AgiBot data. These component mixtures define skill kinship and select prior adapters to initialize new skills. The paper evaluates transfer rankings across four policy families, composition stability on held-out tasks, and continual learning in simulation and on a dual-arm robot.

Critical strengths

I like that the representation is checked against actual policy transfer: across 144 directed pairs, local motion reaches a mean within-target correlation of 0.376, with a 95% interval of [0.185, 0.551] (Table 8). The quantizers and basis are fitted on external AgiBot data without transfer outcomes, which makes the evidence more convincing.

The paper connects this analysis to a practical intervention: ReL raises final simulation success from 16.4% to 26.0% against a random-source baseline using the same adapter framework (Table 14). The held-out composition checks and real-robot experiment add useful support.

Critical weaknesses

I'm not convinced that the discovered primitives improve source selection beyond the original motion distributions. In Table 12, composition alone has a lower point estimate than raw distributions (0.372 versus 0.429); the best result uses fusion, while ReL uses composition alone and lacks a raw-motion retrieval control.

The continual-learning comparison uses one fixed task order in each setting and one sampled Random source history, with no repeated-run uncertainty reported (App. E). On the robot, 32% versus 24% is only four additional successes out of 50 final trials per method, so the reliability of that gain is still unclear.

Questions for authors

How does ReL compare with raw-motion similarity used for source selection in the same adapter framework? If they perform similarly, I'd view the NMF basis mainly as an interpretability tool.

Does the simulation advantage persist across task orders, training seeds and independent Random source draws? Repeated paired runs would materially strengthen my confidence in the practical result.

2 comments

W1 & Q1: Significance of primitives beyond raw distribution.

First, we agree with the reviewer that primitive compositions and raw-motion distributions achieve comparable effectiveness in source selection. However, our primary motivation for extracting primitives is to investigate the shared structure underlying the transfer-relevant representation. The discovered primitives reveal recurring behavioral constituents, make overlapping relationships among skills explicit, and provide a consistent basis for characterizing unseen skills. Their scientific value lies in uncovering how diverse skills are organized through shared components and distinct compositions, with downstream source selection serving as one application of this structure.

Second, we further evaluate raw-motion- and fusion-guided source selection on the original ten-task sequence. Raw-motion similarity and primitive composition select the same source in 7 of 9 decisions. The resulting final success rates show that primitive composition preserves effective source-selection information, while fusion provides an additional empirical improvement.

Source-selection methodFinal success rate
Raw-motion similarity25.2%
Primitive composition26.0%
Fusion27.6%

The transfer analyses provide complementary evidence. Across the 24 target tasks, composition-selected sources yield a mean transfer gain of 14.25%, compared with 13.58% for raw motion. Taken together, these results support composition as an effective source-selection criterion that also exposes the shared behavioral structure underlying skill relationships.

W2 & Q2: Experiments on randomness and different task orders.

Thank you for this suggestion. We provide detialed results on different task orders, random source selection, and random seeds.

1. Different task order.

We evaluated two additional orders of the same ten tasks, keeping the training budget and evaluation protocol fixed across methods. ReL outperforms Random in both additional orders, by 12.8% and 9.6%, complementing the 9.6% gain reported for the original order. The advantage over random-source initialization persists across all three evaluated orders, which support the robustness of relation-guided source selection beyond the original task sequence.

Task orderReL Final (%)Random Final (%)CLARE Final (%)SeqFT Final (%)ReL − Random (pp)
Original order (submitted paper)26.016.417.211.6+9.6
Additional order 120.88.015.29.6+12.8
Additional order 224.414.815.214.8+9.6

2. Different random sequence for the baseline.

To assess the robustness of the Random baseline to source-selection history, we supplement the original history (H1) with two additional histories (H2 and H3), all fixed before training, while keeping the task order and optimization seed unchanged. Random source selection yields limited and inconsistent gains across these inheritance histories, with improvements on some tasks offset by declines on others. These results highlight the importance of characterizing transfer relationships to guide prior-skill selection.

Random selection historyReproduction seedFinal (%)Mean (%)
H1 (original)44429816.407.57
H2 (additional)14749418.809.24
H3 (additional)22846517.208.68
Average—17.47 ±\pm 1.228.50 ±\pm 0.85

3. Repeated Run.

The original experiments used a training seed of 42. We additionally train and evaluate ReL and Random using seeds 43 and 44, and report the mean and standard deviation across these additional runs. ReL outperforms Random with both seeds, yielding an average improvement of 10.93% in the Final score. These results provide additional evidence that the benefit of relation-guided initialization persists across different repetitions.

Training seedReL — Mean (%)ReL — Final (%)Random — Mean (%)Random — Final (%)
4213.0126.007.5716.40
4315.6827.607.3214.40
4414.2624.409.8014.40
Average14.32 ±\pm 1.3426.00 ±\pm 1.608.23 ±\pm 1.3715.07 ±\pm 1.15

4. Robustness on real deployment.

To further evaluate the robustness of our results on the physical robot, we conducted four additional runs, bringing the total number of runs to five. The results are summarized below, with values reported as mean ± sample standard deviation across the five runs. ReL achieves an average improvement of 8.0% over Random. These repeated evaluations provide further evidence that the practical benefits of relation-guided skill reuse are consistent across repeated physical-robot trials.

MethodFinal (%)
CLARE16.00 ± 2.00
Random23.20 ± 2.28
ReL31.20 ± 3.35
Reviewer_WxT3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present Robotaxonomy, a framework for looking at relationships between robot skills and examining the ones that are relevant for transfer (constructing skill relationships from task trajectories) and how it affects transfer. The authors do this by measuring success rate gains for cross-skill adaptation, and they construct skill relationships across various levels of abstraction (task attributes, trajectories, and motion distributions for similar trajectory features). The authors discover that relationships from local motion distributions align closer (strongest correlation) with the transfer effects. The authors also define Skill Kinship, where each skill is represented through a composition across behavior primitives (nonnegative factorization to motion distributions). The authors then practically also initialize new skills from existing skills with the help of Skill Kinship (relation guided learning).

Critical strengths

It is nice that the authors approach the motivation for the representation of the relationships from the standpoint of the abstract levels: comparing the task-level descriptions vs pointwise trajectories vs local motion distributions, and then the authors ground with measured treasured effects (representations are measured transfer outcomes and not just based on intuition, and are able to clearly show that through a comparison of the three abstraction levels).

The authors look carefully at the skill kinship/primitivate composition (in section 4.2), and demonstrate a consistent high kinship ranking agreement above 0.97, and demonstrate high rankings in held out tasks as well.

The authors test performance downstream on through relation-guided learning, and it shows that since the skill kinship composition is good/robust though showing that it outperforms choosing one at random, they compared on both simulation and on a real robot.

Critical weaknesses

The experiments highlight that local motion distributions achieve the strongest correspondence with source rankings and have a Spearman correlation of 0.376 (compared to task-level descriptions, pointwise trajectories, and local motion distributions). Is there any uncertainty/confidence for this number? I see a couple of confidence intervals from the appendix, it would be good to move them forward into the main paper (and it wouldn’t take up much additional space). Also, the correlation seems quite low (although it is higher than the other two representations), which may indicate that there are other aspects that will perform better than just local motion distributions?

The authors claim that motion is what transfers between robot skills (local motion distributions have the most gains compared to task-level descriptions and pointwise trajectories). However, it has been demonstrated generally that language is useful as a prior for transfer. Two wiping skills can move similarly yet involve different surfaces, while skills with different verbs can still share useful motion. The paper treats task labels as too vague, but it never checks whether they add information beyond motion. It would be interesting to look at (after looking at these three representations in isolation), if various combinations of the three would have an even higher correlation? A number of works [1, 2, 3] look at object centric and natural language descriptions for robotic skills/manipulation, and deem them still to be important; and [1] shows that it is easier to transfer verbs for objects that are similar to each other in language space, vs objects that are extremely different, which could mean that physical properties or language attributes in combination of the motion may be highly correlated/useful priors together for transfer.

The paper is framed by talking about transfer for robot skills in general, however the evidence/experiments are largely based around testing/or reusing skills from VLAs/large pretrained policies (Pi, Groot, etc). While VLAs are commonly used as policy, the authors should either narrow down the paper scope to specifically VLA policies or test on various other non-VLA architectures, and also means that the “tasks/motions/skills” that are primarily of focus here are specifically what is often found in the pretraining datasets.

It would be nice for the authors to include a brief description of limitations (there is future work mentioned) in the conclusion and discussion section.

[1] Ma, R., Lam, L., Spiegel, B. A., Ganeshan, A., Patel, R., Abbatematteo, B., ... & Konidaris, G. (2023, October). Skill generalization with verbs. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5844-5851). IEEE.

[2]Wen, J., Zhu, Y., Zhu, M., Li, J., Xu, Z., Che, Z., ... & Tang, J. (2024, May). Object-centric instruction augmentation for robotic manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4318-4325). IEEE.

[3] Yu, H., Kamat, V. R., & Menassa, C. C. (2025). Generalizable skill learning for construction robots with crowdsourced natural language instructions, composable skills standardization, and a large language model. ASCE OPEN: Multidisciplinary Journal of Civil Engineering, 3(1), 04025014.

Questions for authors

Does the transfer hold for multiple non-VLA architecture models/policies? ReL utilizes Pi 0.5 as the base policy, would the claims still hold if other models are used as the base policy? Would be interested to see more analysis into the results that looks into relationship between language descriptions and motion. (Language is still a useful prior overall that could be leveraged, and can be a prior to the motion). Maybe if controlled along similar tasks/different task hierarchy levels?

5 comments

W1:Uncertainty and magnitude of correlation.

Thank you for raising this point. We report the uncertainty of the correlation directly in the main text of the revised manuscript: ρ = 0.376, with a 95% target-bootstrap confidence interval of [0.185, 0.551]. To help interpret its magnitude, we provide numerical context from related task-representation studies and examine the practical gains from source selection.

On the one hand, related studies report correlations of a similar magnitude. Wallace et al. [1] report mean per-target Spearman correlations of 0.35 for Task2Vec in expert selection. Task2Box [2] reports Spearman correlations of 0.39 for its 5D representations when predicting transfer affinities involving novel tasks on Taskonomy. These results provide relevant context for the magnitude of our observed association on more complex scnarios (robotic tasks).

On the other hand, Table 11 demonstrates the practical benefit associated with this correlation: motion-based source selection yields a mean transfer gain of 13.58%, compared with 20.17% for ground-truth selection of the best observed source. It therefore achieves nearly 70% of the maximum gain available.

Reference

[1] Bram Wallace, Ziyang Wu, and Bharath Hariharan. Can We Characterize Tasks Without Labels or Features? CVPR, 2021.

[2] Rangel Daroya, Aaron Sun, and Subhransu Maji. Task2Box: Box Embeddings for Modeling Asymmetric Task Relationships. CVPR, 2024.

W2 & Q3: More analysis on relationships between language and motion.

Thank you for this valuable suggestion. Linguistic relationships at different levels of granularity are important aspects to consider when characterizing transferability between robotic skills. Following your suggestion, we conducted three additional analyses, which have been incorporated into the revised manuscript.

1. Comparison on different semantic granularities.

Starting from the operation name, we progressively add three types of information from the original task instructions:

  • Object roles: the object or surface acted on, the tool used, and the destination or reference object.
  • Functional steps: the actions involved in carrying out the operation and their order.
  • Goals and constraints: the desired final state, location, orientation, and specified execution requirements.
Semantic levelCumulative informationSpearman ρ
L0Operation name0.034
L1L0 + object roles0.120
L2L1 + functional steps0.217
L3L2 + goals/constraints0.179

The description combining operations, object roles, and functional steps (L2) reaches a correlation of 0.217, compared with 0.034 for operation names alone and 0.120 after adding object roles. Further adding finer-grained goals and constraint types does not increase the correlation. These observations suggest an intermediate semantic scale suited to characterizing skill relationships. In the present analysis, affordance-level refinement is more effective for describing transfer relationships as a proper granularity for semantic representations.

2. The effectiveness of fusion on different representations.

For each target task, we combine the normalized motion and language rankings. Under a shared fusion protocol, object roles and functional steps yield the largest observed improvements over motion alone. The full weight sweep further shows that L2 achieves its highest observed correlation of 0.439 at a semantic weight of approximately 30%, compared with 0.376 for motion alone. These results indicate that incorporating semantic information at an appropriate level of granularity and weight can improve the characterization of transfer relationships. They also reinforce that local motion remains the most informative individual representation, while highlighting the potential of combining complementary representations.

Semantic informationFused Spearman ρΔρ95% interval for Δρ
L0: operation0.366−0.010[−0.052, 0.031]
L1: + object roles0.405+0.030[0.004, 0.058]
L2: + functional steps0.418+0.042[0.013, 0.073]
L3: + goals/constraints0.391+0.015[−0.019, 0.049]

3. Relationships between language and motion.

We further clarify that motion distributions can capture behavior-related information that is also expressible in language. The additional t-SNE visualization included in the revised manuscript (Appendix F.4, Figure 10) illustrates semantically interpretable commonalities captured by motion distributions, such as cross-arm object transfer and tool-tip path following. These examples demonstrate that behavioral similarities can reflect semantic relationships grounded in motion dynamics, consistent with the view of language as an abstraction of behavior.

W3 & Q1 & Q2: Comparison on no-vla policies.

On the one hand, we conduct additional experiments with multiple non-VLA policies (ACT [1], Diffusion Policy [2], and BC-RNN [3]) under the same task order. ReL, which uses inter-skill transfer relationships as a prior, achieves consistent improvements over Random, which randomly inherits a previously learned skill.

Base policyMetricReLRandomCLARE
ACTMean11.489.946.66
Final14.4010.404.80
Diffusion PolicyMean19.8518.2119.64
Final21.220.419.2
BC-RNNMean4.010.771.17
Final9.61.64.8

On the other hand, we validate the effectiveness of the local motion distribution on a non-VLA policy, ACT, in Figure 4 (d) (ρ\rho = 0.209). These results indicate that the conclusions are also effective for conventional imitation-learning policies without pre-training. Meanwhile, we have also clarified in the revised manuscript that motion-based relationships remain useful beyond VLAs, while this correlation becomes weaker on non-VLA policies. This difference may reflect the broader repertoire of reusable execution patterns acquired through large-scale robot-policy pretraining.

W4: Limitation paragraph.

Thank you for this suggestion. We have added an explicit discussion of two limitations on different embodiments and modalities in the revision, both of which point to valuable directions for future research.

Reference

[1] Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. RSS, 2023.

[2] Cheng Chi, et al. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. RSS, 2023.

[3] Ajay Mandlekar et al. What Matters in Learning from Offline Human Demonstrations for Robot Manipulation. CoRL, 2022.

Reviewer_yYr2OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This work aims to find a representation that facilitates skill transfer and use that for continual learning of real-world robots. The key idea is to represent each skill as local motion distributions and find representative bases. The authors introduced five measurements for identifying the local motion distributions and a way to extract the bases from the aggregated representations. The local motion distributions are compared with other skill representation methods, which are task-level descriptions and pointwise trajectories. The result shows that the proposed local motion distribution is the representation which most explains the transferability of skills. And a relationship structure called Skill Kinship is proposed so that the skills are decomposed using the identified behavioral primitives.

Critical strengths

This work proposed a systematic framework that incorporates constructing a representation and a skill relationship structure. And it also proposes a simple method to leverage the structure to facilitate the skill transfer, which the authors show effective in both simulated and real-world environments.

Critical weaknesses
  • The proposed representation (the local motion distribution) is indeed a useful instance of summarizing statistics of the trajectory and skills, but it is made up of a set of heuristic measurements. So I think the impact on the community is limited. There are works like Mete et al. (NeurIPS, 2024) that tried to solve the problem of discovering skill abstraction in a self-supervised manner. I think this work and likes should be discussed.
  • The key idea is to decompose the skill (one task) into multiple window statistics, and the method again aggregates these local distributions into an averaged skill distribution that does not account for the temporal structure of the data. Temporal structure itself conveys useful information and there are works that exploit this structure to find useful representations like Bagatella et al., "TD-JEPA" (ICLR, 2026), even though the exact problem setting is different from what this paper discusses.
  • There are a lot of moving parts for the measurements and quantization of the local motion distributions. And for that design complexity, the absolute success rate of the tasks still looks low. In addition, the evaluation protocol for the continual learning experiment in Figure 7 is unclear. Appendix E does not describe how many seeds are used and how the random sequence for the Random baseline is chosen, and there is no variance marked, so I could not check whether the results are statistically meaningful. Furthermore, the Random baseline is much more prone to the randomness of the seed, since it randomly selects which skill to use as an initialization for the next task.
Questions for authors
  1. What were the grounding intuition or theory the authors had in mind when designing the five window measurements of the local motion distribution? What strengths/weaknesses does it have compared to the method that discovers such representation in a self-supervised manner like Mete et al. (NeurIPS, 2024)?
  2. Is there a reason the authors did not consider temporal structure along with the window statistics? If there could be a discussion whether incorporating the temporal structure into consideration would help or not, it would help position this paper onto the literature.
  3. Are the results in Figure 7 statistically significant? The paper would benefit from providing how random seeds affect the performance of the proposed and baseline methods.
7 comments

W1 & Q1: Grounding theory and comparison with self-supervised method.

Thank you for this suggestion. In the following, we clarify the theoretical foundations of our representation design from the perspectives of motion and control science, and extend our discussion of related self-supervised approaches through direct comparisons with QueST and OAT. These additional analyses further support our central finding that local behavioral characteristics exhibit the strongest empirical correspondence with skill transferability.

1. Grounding theory.

Our organization follows the structure of rigid-body motion and multi-DOF control. Vochten et al. [1] distinguish motion magnitude from directional evolution and define geometric progress through accumulated translational and angular motion. This first motivates retaining the amount of translation and rotation separately, since they have different units, and then describing how motion is organized: equal cumulative amounts can arise from directed progression or repeated reversals. Zhai and Milgram [2] distinguish coordination within translation, within rotation, and between the two. This adds a second requirement: the organization of each component does not determine their joint execution. We therefore retain the relative translation–rotation scale and their synchronous magnitude variation, with cross-component correlation providing an established measurement precedent. These principles from motion and control provide the basis for the division of the measurements in Eq. (2) and supply physically motivated priors for their numerical design.

2. Comparison with self-supervised manner.

Self-supervised tokenization methods, such as QueST [3] and OAT [4], provide relevant alternatives for constructing local action representations. We trained from the official implementations using three training seeds (0, 1, and 2). These results reinforce our finding that local behavior provides an informative granularity for characterizing skill transfer. Meanwhile, both self-supervised methods lower down the correlations compared with the representation organized around execution characteristics. This suggests that reconstruction-based methods may leave some transfer-relevant execution properties implicit, whereas our design makes these properties explicit in skill comparison.

Local representationSpearman correlation95% bootstrap interval
Ours0.3759[0.1872, 0.5529]
QueST token distributions0.2506[0.1025, 0.3972]
OAT token distributions0.1777[−0.0002, 0.3522]

3. Strength and weakness.

Our representation measures similarity through shared motion scale, progression, and coordination, while remaining invariant to fixed world-frame transformations. This allows common execution characteristics to establish skill relatedness despite differences in detailed trajectories. In contrast, QueST and OAT learn reconstructable action representations through self-supervised objectives. Our representation instead prioritizes physically interpretable local behavioral characteristics over trajectory reconstruction, introducing an explicit execution-based inductive bias for estimating transfer-relevant skill relationships.

Reference

[1] Maxim Vochten, Ali Mousavi Mohammadi, et al. Invariant Descriptors of Motion and Force Trajectories for Interpreting Object Manipulation Tasks in Contact. TRO, 2023.

[2] Shumin Zhai and Paul Milgram. Quantifying Coordination in Multiple DOF Movement and Its Application to Evaluating 6 DOF Input Devices. CHI, 1998.

[3] Atharva Mete, Haotian Xue, et al. QueST: Self-Supervised Skill Abstractions for Learning Continuous Control. NeurIPS, 2024.

[4] Chaoqi Liu, Xiaoshen Han, et al. OAT: Ordered Action Tokenization. RSS, 2026.

W2 & Q2: Temporal structure.

Thank you for raising this point. We need clarify that the original representation have already captured short-term temporal information. Meanwhile, we directly evaluated whether adding global temporal information improves relationship estimation, and expanded the related works to cover temporal modeling approaches such as TD-JEPA [1] in the revised manuscript.

1. The original representation have retained short-term temporal relationships.

The window measurements are computed from ordered pose sequences. The reversal statistic η\eta uses adjacent motion directions, while CC captures synchronous variation between translation and rotation magnitudes. These measurements can distinguish directed from back-and-forth progression and simultaneous from sequential translation–rotation activity, even when cumulative motion is matched. The representation therefore captures aspects of how local motion unfolds, alongside its magnitude and spatial organization.

2. Direct evaluation of incorporating global temporal information.

To investigate whether retaining temporal information improves transfer relationship estimation, we evaluate a coarse-grained global temporal extension. Specifically, each window is assigned to an early, middle, or late phase based on its midpoint relative to the full demonstration, and this phase label is incorporated into the original feature distributions. We then compare two matching strategies: corresponding-phase matching, which enforces temporal alignment, and permutation-invariant phase matching, which retains phase-specific information while allowing phase correspondence to vary.

RepresentationMean Spearman correlation95% target-bootstrap interval
Original aggregation0.3759[0.1872, 0.5529]
Corresponding-phase matching0.2793[0.1225, 0.4342]
Permutation-invariant phase matching0.3089[0.1305, 0.4808]

The original aggregation achieves the highest point estimate, while relaxing phase correspondence partially recovers the correlation achieved by the temporal extension. This pattern suggests that reusable local execution patterns may occur at different stages across tasks, such that retaining their global temporal positions can constrain the identification of shared behavioral structure. These results therefore support our choice to preserve short-timescale execution patterns while aggregating across task stages for relationship estimation.

3. The role of temporal structure depends on the representation's objective.

TD-JEPA learns policy-conditioned, multi-step latent predictions for zero-shot reinforcement learning, with future-dynamics modeling as a central learning objective. In contrast, our relationship model aims to identify shared execution experience for cross-skill adaptation, allowing common local behavioral patterns to contribute to skill relatedness even when their task-level temporal arrangements differ. This analysis provides empirical support for using local execution patterns as the primary basis for relationship estimation and highlights the distinction between our objective and representations designed for long-horizon prediction.

Reference

[1] Marco Bagatella, Matteo Pirotta, et al. TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning. ICLR, 2026.

W3 & Q3: Statistical significance of results.

Thank you for raising this point. We provide additional results on different seeds and random sequences to further support statistical significance of results.

1. Clarify evaluation protocal in Figure 7.

We have clarified in the revised caption of Figure 7 that, after each training stage, we evaluate all tasks learned so far and report their average success rate in simulation, with a direct reference to the evaluation-protocol paragraph in Appendix E.1.

2. Experiments on additional seeds and variance.

The original experiments used a training seed of 42. We additionally train and evaluate ReL and Random using seeds 43 and 44, and report the mean and standard deviation across three different runs. ReL achieves an average Final-score improvement of 10.93% over Random. These results support the statistically significance of results in Figure 7.

Training seedReL — Mean (%)ReL — Final (%)Random — Mean (%)Random — Final (%)
4213.0126.007.5716.40
4315.6827.607.3214.40
4414.2624.409.8014.40
Average14.32 ±\pm 1.3426.00 ±\pm 1.608.23 ±\pm 1.3715.07 ±\pm 1.15

3. Different random sequence for the baseline.

We use name-and-date strings as seeds for a uniform sampler in selecting source of the Random baseline. To assess robustness to random source selection, we supplement the original selection sequence (H1) with two additional random sequence (H2 and H3). Random source selection yields limited gains across different inheritance histories, with improvements on some tasks offset by declines on others, highlighting the importance of characterizing transfer relationships to guide the choice of prior skills. We've provided detailed random source selection sequence in the revision (Table 22, Appendix E.4.).

Random selection historyReproduction seedFinal (%)Mean (%)
H1 (original)44429816.407.57
H2 (additional)14749418.809.24
H3 (additional)22846517.208.68
Average—17.47 ±\pm 1.228.50 ±\pm 0.85

Comments on the paper

2 comments

Dear Reviewers,

Please review the authors' rebuttal and the other reviewers' comments, and participate in the discussion before it closes on November 18 (AoE). Kindly indicate whether your concerns have been adequately addressed and raise any remaining questions or concerns.

Thank you for your time and contributions.

Best regards,

Area Chair

Dear Reviewers,

A quick reminder that the discussion period ends on November 18 (AoE). If you haven't already, please review the authors' rebuttal and provide a brief follow-up indicating whether your concerns have been adequately addressed or if further clarification is needed.

Your timely feedback would be greatly appreciated and will help facilitate the final decision.

Thank you for your time and contributions!

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.