- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposed an unified benchmark environments for Cross-domain Reinforcement Learning (CDRL) - which includes 1). Same-task morphology transfer, 2). cross-task sub-skill transfer, 3). transfer from pretrained generalist models, and 4). real-world transfer on physical robotic hardware; Then the authors also provide comprehensive comparisons among existing work and make lots of analysis over it; Overall - this paper is well-written and easy to follow, and it definitely has the contributions to the community. It also covers lots of environments in the previous work and also include the real-world transfer which is more practicals in my opinion. I prefer to accept it; The only concern is should this paper be classified into Evaluations & Datasets Track?
Critical strengths
- Provide an unified benchmark environments for the CDRL research community.
- Have comprehensive comparison among several existing methods.
- Provide the setting for the real-transfer in the real physical robotic, and the robot is also the open-sourced one which allow easy access for other researchers
Critical weaknesses
- The user didn't report the success rate, only the accumulated reward is reported, which cannot reflect if the agent complete the task or not;
- The number of tasks is not that lot, of course users has proposed a very new setting, which is real-transfer, however, for the other tasks/settings they're already exists in the previous literature reviews; Although it's not a big problem, it still recommend the authors can keep adding more meaning tasks into the environment.
- This work is more suitable for Evaluations & Datasets Track?
Questions for authors
The only question is - will the authors keep maintaining the benchmark repo - for example - adding more robotic arm selections, adding more tasks etc, and keep adding more baselines. This can help other researchers to focus on the core algorithms developments and avoid repeating baseline training work.
2 comments
Dear Reviewer Lheo,
Thank you for your positive and encouraging review. With about 24 hours left in the discussion, we kindly invite you to review our rebuttal and are happy to clarify any remaining concerns. We briefly summarize the main updates addressing your concerns:
Success rates (Q1): We added success-rate curves for all task-oriented environments, which closely track returns, while retaining returns to capture incremental transfer progress. Representative results for cross-morphology transfer on Door/Lift are shown in results, with the remaining results in Appendix C.
Benchmark coverage and expansion (Q2): We clarified that CDRL-Bench unifies 53 transfer configurations under shared protocols, baselines, and source models, and added a Baxter-to-GR1 humanoid transfer experiment.
Maintenance (Q3): We committed to continually expanding the benchmark with new tasks, embodiments, and baselines, and to maintaining reusable implementations and pretrained source models to support future research.
Venue fit (Q4): ICLR 2027 has no separate Datasets & Benchmarks track, and its Call for Papers explicitly includes datasets and benchmarks. Beyond the resource itself, our evaluation offers insights into when transfer helps or hurts across embodiments, when additional sources fail, and how inherited critics impede fine-tuning.
We hope these additional experiments and clarifications have addressed your concerns and would be grateful if you could take them into account in your final evaluation. We sincerely appreciate your time and valuable suggestions, and look forward to hearing from you.
Best regards,
Authors of Paper 18680
Response to Reviewer Lheo
We sincerely thank the reviewer for the thoughtful and constructive feedback. We address each concern point by point below and have revised the manuscript accordingly, with all changes highlighted in blue.
A: Thank you for the suggestion. We agree that cumulative reward alone may not fully reflect whether an agent successfully completes a task. To address this, we additionally include success-rate learning curves alongside reward curves for all task-oriented environments. Due to space constraints, we only show the results of cross-morphology transfer on the Door/Lift tasks (see: https://imgur.com/a/Wlm6yOx, corresponding to Figures 17-18 in Appendix C). Evaluation rewards and success-rate curves for all remaining task-oriented environments are provided in Appendix C.
The success-rate trends closely follow the evaluation rewards: as returns approach expert-level performance, success rates approach 1.0. Conversely, for sub-skill and navigation tasks where rewards show limited improvement, success rates remain near zero. For locomotion tasks, where binary success criteria are not defined, we continue to report episodic returns.
Episodic return is particularly important for evaluating knowledge transfer in CDRL, as it captures incremental learning progress that binary success rates may overlook. Since the primary goal of CDRL is to assess how effectively transferred knowledge accelerates target-domain learning, relying solely on success rates could obscure meaningful improvements before task completion. We therefore report both episodic returns and success rates (where applicable) to evaluate transfer efficiency and task completion comprehensively.
A: Thank you for the suggestion. CDRL-Bench covers 53 transfer configurations spanning locomotion, robotic manipulation, maze navigation, cross-task sub-skill transfer, and Real2Real transfer, varying in embodiment, task, source composition, and observation modality.
We agree that several individual settings have precedents. The contribution lies in bringing these previously fragmented settings into a unified evaluation framework, alongside Real2Real transfer. Shared protocols, baselines, and pretrained source models enable systematic comparisons that isolated evaluations cannot provide, revealing how transfer effectiveness changes across morphology families, target embodiments, and source combinations.
We have also added GR1 humanoid → Baxter transfer in Robosuite, expanding coverage to substantially different embodiments. We will continue adding meaningful tasks, embodiments, and baselines. In summary, CDRL-Bench combines established and new settings to test whether CDRL advances generalize beyond the individual tasks on which they were developed.
Thanks for the suggestion! Yes, we are committed to maintaining CDRL-Bench as a shared resource for the CDRL community. Specifically:
We will expand its robotic embodiments, transfer tasks, and baseline algorithms. Notably, the newly added GR1 → Baxter scenario and morphology-aware baselines are concrete steps in this direction.
We will also maintain reusable baseline implementations and pretrained source models, with documented training and evaluation configurations, to support reproducible comparisons.
Our goal is to let researchers build on a common evaluation infrastructure and devote their effort to algorithmic advances rather than repeatedly implementing and training baselines.
A: Thank you for raising this question. We agree that the work has a strong evaluation and benchmarking focus. Our understanding is that ICLR 2027 does not offer a separate Datasets & Benchmarks track, while its Call for Papers explicitly includes “datasets and benchmarks” among the relevant topics. We therefore believe this submission aligns with the conference’s scope.
Beyond the benchmark itself, our unified evaluation reveals when transfer helps or hurts across embodiments, when additional sources fail to improve performance, and how inherited critics can impede fine-tuning.
We hope CDRL-Bench contributes both a useful community resource and empirical insights that support the development of more general CDRL methods.