- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this work the authors inspect model-free, Dyna-style model-based, and model-based planning agents in cross-domain adaptation. They identify initializing the target critic from scratch as a weak-point in model-free adaptation, and the interaction between environment dynamics model accuracy and the prior guided data collection as a limiting factor in model based methods. These motivate their improved approach - "DOMAINADEPT" which synthesizes these findings and improves adaptation.
Generally my view of the paper is positive, while the evidence for the generality of its claims can be strengthened.
Critical strengths
-Important contribution towards characterizing the contribution of different components of different RL approaches in cross-domain transfer -Insights are synthesized and motivate DOMAINADEPT - which combines dynamics ensembles and proposal policy tethering - improving performance on evaluated tasks. -The authors test the main findings on 10 random seeds.
Critical weaknesses
-The paper evaluates multiple tasks, in which each manipulation task uses a single fixed target dynamics configuration. Consequently, the experiments do not establish how consistently the findings hold across different magnitudes and combinations of dynamics shifts within a task. This limits the generality of the conclusions, particularly because the tethering coefficient is selected separately for each task.
Questions for authors
- Can the authors provide evidence that the main findings and DOMAINADEPT's gains hold across multiple target dynamics configurations within each task?
2 comments
Follow up on our responses
Dear Reviewer 7kGB,
With two days remaining in the discussion period, we wanted to follow up on our response. To address your question on generality, we evaluated DomainADEPT on six target dynamics configurations of Threading, varying the magnitude and combination of shifts (including an actuator delay), using a single tether coefficient without per-configuration tuning. DomainADEPT outperforms SimDist and IBRL in all six configurations.
We would appreciate it if you could let us know whether this addresses your concern. If so, we would be grateful if you would consider updating your score; if not, we would welcome any guidance on what further evidence would be most helpful. Thank you for your time.
Reply to Q1, W1: DomainADEPT gains across multiple target dynamics configurations with single tethering coefficient
We thank the reviewer for this point. Given the limited rebuttal period, we focus on one task, Threading, and run a new experiment that tests within-task shifts on this task. We found that DomainADEPT outperforms several target dynamics configurations with the same tethering strength on one task. All methods use a single, untuned tether coefficient λ=1 (10 seeds, 50k target interactions). We will add the same kind of multi-configuration study for the remaining tasks in the revision.
We have different configurations and types of shifts for this task:
We evaluate DomainADEPT comparing with its baselines: SimDist and IBRL in these several configurations:
It shows DomainADEPT leads in several different dynamic shift configurations. We also report where performance degrades. Absolute performance falls as the shift grows.