acceptodds
Under review as a conference paper at ICLR 2027

Demystifying Cross-Domain Policy Adaptation with World Models

Abstract

Cross-domain policy adaptation seeks to reuse source-domain knowledge under dynamics shifts with limited target-domain interaction. Model-based methods address this problem by adapting source-pretrained world models, but their apparent sample-efficiency advantage may partly arise from value transfer: model-based methods often retain source-trained value functions, while model-free baselines learn critics from scratch. When we give model-free baselines the same source-trained critic, they initially adapt faster than their Dyna-style counterparts. Further analysis identifies that dynamics ensembles and prior-guided data collection jointly improve Dyna-style adaptation, allowing it to outperform the model-free baseline on average. Guided by these findings, we propose \DomainADEPT, which improves model-based planning both with a dynamics ensemble to improve its accuracy and with “tethered” proposal policy fine-tuning to limit policy drift during data collection. Across four contact-rich manipulation tasks and one locomotion task, \DomainADEPT achieves the highest average final performance among the evaluated methods under matched target-domain interaction budgets, improving average success by 25 percentage points over the strongest baseline.

Primary area:
reinforcement learning
Keywords:
Model-based reinforcement learning, Cross-domain adaptation, Dynamics gap

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_7kGBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work the authors inspect model-free, Dyna-style model-based, and model-based planning agents in cross-domain adaptation. They identify initializing the target critic from scratch as a weak-point in model-free adaptation, and the interaction between environment dynamics model accuracy and the prior guided data collection as a limiting factor in model based methods. These motivate their improved approach - "DOMAINADEPT" which synthesizes these findings and improves adaptation.

Generally my view of the paper is positive, while the evidence for the generality of its claims can be strengthened.

Critical strengths

-Important contribution towards characterizing the contribution of different components of different RL approaches in cross-domain transfer -Insights are synthesized and motivate DOMAINADEPT - which combines dynamics ensembles and proposal policy tethering - improving performance on evaluated tasks. -The authors test the main findings on 10 random seeds.

Critical weaknesses

-The paper evaluates multiple tasks, in which each manipulation task uses a single fixed target dynamics configuration. Consequently, the experiments do not establish how consistently the findings hold across different magnitudes and combinations of dynamics shifts within a task. This limits the generality of the conclusions, particularly because the tethering coefficient is selected separately for each task.

Questions for authors
  • Can the authors provide evidence that the main findings and DOMAINADEPT's gains hold across multiple target dynamics configurations within each task?
2 comments
AuthorsOpenReview ↗

Reply to Q1, W1: DomainADEPT gains across multiple target dynamics configurations with single tethering coefficient

We thank the reviewer for this point. Given the limited rebuttal period, we focus on one task, Threading, and run a new experiment that tests within-task shifts on this task. We found that DomainADEPT outperforms several target dynamics configurations with the same tethering strength on one task. All methods use a single, untuned tether coefficient λ=1\lambda = 1 (10 seeds, 50k target interactions). We will add the same kind of multi-configuration study for the remaining tasks in the revision.

We have different configurations and types of shifts for this task:

Dynamics shift configurationobject massobject frictionfixture frictioninsertion clearancegripper stiffnessmove center of mass centeraction delay step
11.70.71.40.920.35––
2 (existing experiments)3.00.52.00.850.12––
35.00.33.00.850.08––
48.00.33.00.850.05––
58.00.33.00.820.05-0.12–
63.00.52.00.850.12–15

We evaluate DomainADEPT comparing with its baselines: SimDist and IBRL in these several configurations:

Dynamics shift configurationDomainADEPTSimDistIBRL
10.9740.7850.685
2 (existing experiments)0.9580.760.67
30.6550.2910.057
40.1870.0570.011
50.180.0490.007
60.6820.3170.134

It shows DomainADEPT leads in several different dynamic shift configurations. We also report where performance degrades. Absolute performance falls as the shift grows.

AuthorsOpenReview ↗

Follow up on our responses

Dear Reviewer 7kGB,

With two days remaining in the discussion period, we wanted to follow up on our response. To address your question on generality, we evaluated DomainADEPT on six target dynamics configurations of Threading, varying the magnitude and combination of shifts (including an actuator delay), using a single tether coefficient without per-configuration tuning. DomainADEPT outperforms SimDist and IBRL in all six configurations.

We would appreciate it if you could let us know whether this addresses your concern. If so, we would be grateful if you would consider updating your score; if not, we would welcome any guidance on what further evidence would be most helpful. Thank you for your time.

Reviewer_MhBXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Cross-domain policy adaptation aims to transfer policies learned in source environments to target domains with different system dynamics and limited interactions. Existing model-based approaches have shown promising adaptation performance by leveraging pretrained world models. However, it remains unclear whether such advantages mainly come from the world model itself or the transferred value functions. In this paper, the authors investigate this issue by disentangling the effects of value transfer and dynamics adaptation. Based on the findings, they propose DOMAINADEPT, which integrates dynamics ensembles and tethered proposal policy fine-tuning to improve planning while preserving the source value function. Experiments on four manipulation tasks and one locomotion task demonstrate the effectiveness of the proposed method in improving adaptation efficiency and task performance under dynamics shifts.

Critical strengths
  1. An interesting contribution of this paper is that the authors investigate where the actual advantages come from. In particular, the finding that source value transfer plays an important role is insightful, and could help us better understand and evaluate existing model-based adaptation methods.

  2. The authors identify dynamics accuracy and prior-guided data collection as two important factors, and introduce dynamics ensembles and policy tethering to address them. This makes the method design more natural and convincing.

  3. The paper provides relatively thorough experiments to investigate different components, including value transfer, dynamics accuracy, and proposal policy adaptation.

Critical weaknesses
  1. One concern is whether the source value function can remain reliable under significant dynamics shifts. Even when the reward function stays unchanged, different dynamics may change the reachability of states and the optimal action choices. The authors show that freezing the source value works well in their experiments, but I am not fully convinced this assumption can generalize to more challenging environments.

  2. The proposed tethering mechanism encourages the adapted policy to remain close to the source policy. While this helps preserve useful behaviors, I wonder whether it could also prevent the agent from discovering better strategies in the target domain. For example, significant changes in friction or contact dynamics may require fundamentally different actions rather than small adjustments to the source behavior. The method seems to assume that the source policy remains sufficiently relevant, which may not always hold.

Questions for authors
  1. Would it be possible to include an experiment with progressively larger dynamics shifts, such as changes in friction or actuator response? I think it would be interesting to see at what point the source value function becomes less reliable, and how this affects the adaptation performance.

  2. The proposed tethering mechanism helps preserve useful source behaviors, but I wonder whether it could also restrict adaptation when the target domain requires substantially different actions. Could the authors discuss how to balance source policy preservation and exploration of new behaviors, especially when the source policy performs poorly in the target environment?

  3. The method uses the mean prediction from multiple dynamics models during planning. I am curious whether this could introduce issues in contact-rich environments, where different models may predict different contact outcomes.

  4. Since the value function depends on the transition dynamics, I wonder whether the authors could further explain why the source value remains effective after adapting the dynamics model. Are there particular properties of the evaluated tasks that make value transfer reliable? I think this would help clarify the conditions under which the proposed method is expected to work.

  5. I think the paper could benefit from a deeper discussion of related studies on cross-domain policy adaptation, including [1], [2], and [3]. These works explore different ways of transferring source-domain knowledge to target environments, such as latent-space adaptation, world-model transfer, and correcting dynamics mismatch. Although some of these studies have already been cited, I would be interested to see a clearer discussion of when adapting the world model is preferable to adapting the policy or its latent representation. This could also help better explain the advantages and limitations of the proposed selective adaptation strategy.

[1] Da et al. Latent Adaptation of Foundation Policies for Sim-to-Real Transfer. ICLR 2026.

[2] Levy et al. Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation. RSS 2026.

[3] Eysenbach et al. Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers. ICLR 2021

6 comments
AuthorsOpenReview ↗

Summary of rebuttal experiments and key results

We thank the reviewer for the thoughtful review and the positive assessment. We appreciate the recognition that identifying where the advantages of model-based adaptation actually come from is insightful, the method design follows naturally from the analysis, and the experiments on each component are thorough. Several questions and weaknesses are closely related, so we group them by topic below. In summary, we have added the following key experiments and will revise the paper accordingly: (i) a ladder of progressively larger dynamics shifts (plus a gripper delay and a rotated hole), together with oracle-dynamics experiments, which shows that the frozen source value remains effective as long as the shift changes how states are reached but not which states are good, and identifies when it breaks; (ii) a tether-strength ablation on shifts that require new behavior, which shows that tethering does not prevent new behavior and that a moderate tether is a robust default; (iii) a per-contact-phase analysis of ensemble predictions, which finds no sign of multimodal contact outcomes and shows that the main source of error is a bias shared across heads, not disagreement between them; and (iv) comparisons with latent adaptation and DARC, which clarify when adapting the world model is preferable to adapting the policy or its latent.

AuthorsOpenReview ↗

Reply to W1, Q1, Q4: Is the source value reliable under larger shifts, and why does it still work?

We thank the reviewer for the careful questions. We have expanded our experimental analysis to answer this question and found that world model adaptation with source value transfer is robust to progressively larger dynamics shifts that significantly change how states are reached but not which states are good. We further identified a dynamics shift that does change which states are good, and find that the transfer works poorly without also adapting the value function.

For the progressively larger dynamics shifts, we list their parameters below:

Difficult levelobject massobject frictionfixture frictioninsertion clearancegripper stiffnessmove center of mass center
Level 01.70.71.40.920.35–
Level 1 (existing experiments)3.00.52.00.850.12–
Level 25.00.33.00.850.08–
Level 38.00.33.00.850.05–
Level 48.00.33.00.820.05-0.12

We also have two other different shifts to change its success behaviour: first, we changed the hole from a square to a diamond, which requires the robot to rotate the needle by 45° to insert it correctly, otherwise the task fails; second, we add a 15-step delay on gripper commands, correspond to “actuator response” suggested by the reviewer. This first change substantially alters the dynamics leading to success, to the point where the source value becomes completely unreliable. To test whether the source value alone is sufficient for planning on this task, we ran an oracle-dynamics + source-value experiment, using the target simulator itself as the dynamics model to do MPPI (analogous to Fig. 2 in the paper). Results are shown below:

SettingsOracle experiment success rate (100 evaluation)Source policy itself
Level 01.00.65
Level 11.00.11
Level 20.980.01
Level 30.880
Level 40.760
45° rotation00
Gripper delay0.960.02

The source value stays effective at Levels 0–3 and gripper delay, even though the source policy itself gradually fails. The Levels 0–3 shifts and the gripper delay shift change the underlying dynamics: they change how states are reached, but not which states are good. MPPI uses the value only to rank candidate states, so the source value remains useful as long as the relative quality of states is preserved. The source value begins to fail once the shift substantially changes which states are good, as in Level 4 and the 45° rotation. Level 4 moves the needle’s center of mass to the tip, which pushes the in-hand needle pose outside the support of the source data. Successful states then typically require the gripper to be lifted higher, and the success rate drops to 0.76. The 45° rotation fundamentally changes the success states, so the states that previously lead to success can no longer succeed at all. In this case, the source value fails completely, and a new target value must be trained in this domain to achieve better performance.

AuthorsOpenReview ↗

Reply to W2, Q2: Does the tether prevent new behaviour?

To test whether tethering prevents the agent from discovering new behaviors, we ran a new ablation over the tether strength (λ∈{0,1,3}\lambda \in \{0, 1, 3\}) on three shifts where the source policy fails and new behavior is required. The key takeaways are that tethering does not prevent new behavior and a moderate tether (λ=1\lambda = 1) performs best or on par with the best, while a strong tether (λ=3\lambda = 3) hurts when the source actions are systematically wrong.

We consider two shifts where the source value remains effective but the source policy fails (Level 3 in [Reply to W1, Q1, Q4] and gripper delay), and one shift where the source actions are systematically wrong (the action frame is rotated by 45°). Each checkpoint is evaluated over 300 episodes after adaptation with a 50k budget (10 seeds).

Settingsλ=0\lambda = 0λ=1\lambda = 1λ=3\lambda = 3
Level 30.1320.1870.185
Gripper delay0.6450.6820.686
Action frame 45° rotation0.5570.5970.493

When the source value remains effective, tethering helps adaptation rather than hindering it: both λ\lambda = 1 and λ\lambda = 3 outperform λ\lambda = 0. Tethering is essentially a regularizer applied during the BC stage before adaptation begins. Without it, the policy drifts away from the source policy into regions where the dynamics model and the source value are inaccurate. As a result, the pre-adaptation MPPI success rate drops from 0.15 (Level 3) / 0.54 (Gripper delay) to 0.05 / 0.09, respectively. Tethering acts only before adaptation starts. During adaptation, MPPI searches a region far wider than the policy’s proposals, so the executed behavior can still deviate from the source policy even with tethering.

When the source behaviour is systematically wrong, an strong tether does hurt, but a moderate tether still provides a small benefit. Under the 45° action-frame rotation, λ\lambda = 3 underperforms both λ\lambda = 0 and λ\lambda = 1, while λ\lambda = 1 is slightly better than λ\lambda = 0.

Reviewer_JSYhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies cross-domain policy adaptation under dynamics shifts with a limited target-domain interaction budget. It first identifies an important confound in prior comparisons: model-based methods retain the source-trained value function while model-free baselines learn critics from scratch. Controlling for this "value transfer", the authors show that model-free adaptation is stronger than previously believed, and that Dyna-style methods only surpass it when a prior-preference margin (which keeps early data collection near the prior) is combined with a dynamics ensemble (which lowers model error). For model-based planning, they find dynamics error is the main bottleneck and that adapting the value head consistently hurts. Based on these findings they propose DomainADEPT (SimDist + dynamics ensemble + tethered proposal fine-tuning), which achieves the best average adaptation performance across five simulated tasks under matched interaction budgets.

Critical strengths
  1. Controlled dissection of a real confound. Table 1 cleanly separates value transfer from world-model adaptation. The observation that a source-critic-initialized IBRL (MF-2) beats the unmodified Dyna-IBRL (DY-1) is a valuable caution for the field: sample-efficiency claims in cross-domain model-based RL have partly compared "value transfer + model" against "no transfer".
  2. Fine-grained, honest analysis. Table 2 shows the margin helps through data collection rather than TD-target construction; Table 3 isolates value adaptation as harmful; Fig. 2's oracle-dynamics experiment shows the frozen source value suffices given accurate dynamics; App. D.1 reports five failed attempts (VAML, pessimistic value ensembles, value calibration, self-distillation, stronger proposals). Negative results are reported rather than buried.
  3. Effective and simple method with consistent gains. DomainADEPT adapts fastest and reaches the highest final success on all five tasks (Fig. 4), the ablation (Fig. 5) supports the two components being complementary, and the recipe transfers to pixel observations (41% vs 31% on Threading). Algorithm 1 and App. C provide reproducible detail (three stages, hyperparameters, latency analysis).
  4. Fair model-based comparison. Dyna-style and planning variants share one distilled world model (end of Sec. 4.1), so the comparison isolates how the model is used rather than which model was trained.
Critical weaknesses
  1. Presentation. The paper contains several concrete defects: the method name is misspelled "DoaminADEPT" in the titles of Algorithm 1 and Table 7; the caption of Fig. 2 reads "trt only" / "src + trt" where "tgt" is meant (the figure legend itself uses "tgt"); the introduction writes "25% percentage points" while the abstract correctly writes "25 percentage points"; and the method name alternates between "DOMAINADEPT" and "DomainADEPT". Beyond these typos, the paper is dense and often hard to follow: the two-channel prior-preference margin (Eq. 1–2, applied to both rollouts and TD targets) would benefit from a small diagram or pseudocode.

  2. Limited method novelty. Both proposed components are established techniques: the dynamics ensemble is a bootstrap ensemble in the spirit of Lakshminarayanan et al. (2017), as the authors themselves acknowledge in App. A.2, and tethered proposal fine-tuning is behavior cloning with output-space regularization toward a frozen source policy (distillation-style; cf. Hinton et al., 2015; Li & Hoiem, 2017). The contribution therefore lies in the analysis that motivates their combination rather than in new machinery.

  3. The value-transfer control is partial. The benchmark cleanly isolates how a shared world model is used (Dyna-style vs. planning), but the model-free vs. model-based axis is only partially equalized. Handing IBRL the source critic transfers one channel of source knowledge, while the model-based pipeline additionally carries a source-trained encoder (frozen throughout adaptation) and value targets relabeled by the source critic QesQ_{es} during world-model training (Eq. 3). The claim in Sec. 1 that the benchmark "isolates value transfer from world-model adaptation" is therefore stronger than the control supports.

  4. Real-robot evidence is preliminary. App. D.5 reports grasp success only and states that no full-task successes were observed in the evaluated trials, with the residual visual sim-to-real gap left unresolved. The authors are transparent about this — the section is titled "Preliminary" — but the consequence is that the real-world claim currently rests on grasp-stage results at small scale (10 trials per point).

Questions for authors
  1. Are the 20 target demonstrations counted against the 50k budget? How does performance change with 5 or 10 demonstrations, and with non-expert demonstrations?
7 comments
AuthorsOpenReview ↗

Summary of rebuttal experiments and key results

We thank the reviewer for the careful and constructive review. We appreciate the recognition of the value-transfer confound we identify, the fine-grained analysis and reported negative results, the complementary components of DomainADEPT and the fair model-based comparison. Below, we address each concern in detail. In summary, we have added the following key experiments and will revise the paper accordingly: (1) Value-transfer control (W3). We added control experiments for value transfer, primarily controlling for encoder transfer, which support and strengthen our conclusions. (2) Real-robot experiments (W4). We completed the real-robot evaluation and compared DomainADEPT to SimDist on two tasks, where our method is better. These sim-to-real results further support our claims. (3) Demonstration ablations. We ran experiments with fewer and lower-quality demonstrations. In these settings, all methods degrade, and DomainADEPT consistently outperforms the baselines.

AuthorsOpenReview ↗

Reply to W3: The value-transfer control is partial

We thank the reviewer for this observation regarding the confounding of the source-trained encoder and critic. We conducted additional experiments showing that the advantage of model-based methods persists even when the source-trained encoder is also transferred to model-free methods. This advantage cannot be explained by encoder transfer alone, and the finding in Sec. 4.1 that DY-4 outperforms MF-3 remains valid under this control. For the source-trained encoder, we equip IBRL with a source-pretrained encoder that is kept frozen during adaptation. Specifically, we consider three variants: (i) Model-based encoder. IBRL is trained on top of the representation from the model-based source-pretrained encoder. The encoder stays frozen, and only the actor and critic are adapted. We use the same encoder EξE_\xi, policy πη\pi_\eta and critic q^ψ\hat q_\psi (correspond to “value targets relabeled by the source critic” mentioned by the reviewer) that Dyna-IBRL/SimDist used in Table 1. (ii) Shared encoder. The actor and critic are trained in the source domain with a shared encoder (learned with actor and critic loss), which is frozen during adaptation. (iii) Separate encoders. The actor and critic are trained in the source domain, each with its own encoder, and both encoders are frozen during adaptation. Here, all IBRL baselines use a prior-preference margin and a source critic (comparable to MF-3 in Table 1).

Method10k20k30k40k50k
IBRL with model-based encoder0.020.0940.1680.3080.378
IBRL with shared encoder0.3290.4910.5030.5230.534
IBRL with separate encoder0.4630.5380.5730.6260.669
IBRL (existing in the paper, MF-3)0.210.3850.4620.650.673
Dyna-IBRL (existing in the paper, DY-4)0.1670.3910.5060.6750.711
DomainADEPT (existing in the paper, ours)0.9150.9540.9390.9710.958

The results show that none of the three encoder variants improves IBRL's performance, and all remain inferior to DY-4 in Table 1 (Dyna-IBRL in the table above). Taken together, they illustrate that transferring a source-pretrained encoder does not improve model-free adaptation: encoders trained with RL losses provide no benefit over MF-3, and initializing IBRL with the world-model components (encoder, policy and critic) performs worse than the native source critic. We hypothesize that the world-model critic head, trained on source-relabeled targets mostly over expert actions, is less reliable for IBRL’s action selection than a TD-trained critic. Thus, our conclusion in Sec. 4.1 remains valid. We will include this more strictly controlled experiment in the appendix.

AuthorsOpenReview ↗

Reply to W4: Real-robot evidence is preliminary

We agree that the real-robot results we originally submitted were preliminary. We have built on these results and completed two real-world tasks with comparisons. Building on our preliminary experiments, we have further improved the real-robot evaluation and plan to add it to the end of the main text. The key takeaways are that DomainADEPT consistently outperforms SimDist in both tasks. DomainADEPT achieves a higher success rate at Round 0, owing to the tethered proposal fine-tuning. As adaptation proceeds, the benefit of the dynamics ensemble becomes evident, and DomainADEPT eventually reaches a success rate close to 100%. We will also update the real-robot demo videos in the supplementary material.

In these experiments, we adapted DomainADEPT and SimDist on the real robot for 5 rounds, collecting 10 trajectories per round. Results on the two tasks are shown below.

Threading

MethodRound 0Round 1Round 2Round 3Round 4Round 5
DomainADEPT12/2015/2016/2019/2020/2020/20
SimDist8/2010/209/2011/2012/2010/20

Square

MethodRound 0Round 1Round 2Round 3Round 4Round 5
DomainADEPT10/2015/2018/2019/2019/2018/20
SimDist9/2011/2013/2012/2014/2013/20

Comments on the paper

2 comments
AuthorsOpenReview ↗

Updated Supplementary Material: Code and Real-Robot Videos

Dear AC and Reviewers,

We have updated the supplementary material with the following:

  • Code. We release the anonymized code for DomainADEPT and the baselines used in the paper, including the training scripts for DomainADEPT, the benchmark environments. Please see the README for instructions.

  • Real-robot videos. We provide videos of the improved real-robot experiments (see our response to Reviewer JSYh) on two tasks, Threading and Square. Both DomainADEPT and SimDist are adapted on the real robot for 5 rounds, with 10 trajectories collected per round. For each checkpoint, the videos show the first 10 of the 20 evaluation episodes. All initial object positions are reset by hand to calibrated positions on the table, so both methods are evaluated from the same set of initial positions.

We hope these materials help with assessing the reproducibility of our results and our real-world claims. Thank you for your time and consideration.

AuthorsOpenReview ↗

Summary of the revised paper

Dear AC and Reviewers,

We thank the reviewers again for the constructive feedback. We have uploaded a revised version of the paper that incorporates the experiments and clarifications from our individual responses. We marked revision by blue text. Below we list what changed and where to find it.

1. Real-robot experiments (new Sec. 6.4, Fig. 6; Reviewer JSYh W4)

  • The preliminary grasp-only results in the appendix are replaced by a full sim-to-real evaluation in the main text, on Threading and Square with a Franka Research 3 robot. The supplementary videos are updated.
  • We compare DomainADEPT with SimDist and IBRL over five adaptation rounds of 10 real-world episodes each, with 20 evaluation trials per point.
  • DomainADEPT reaches 20/20 on Threading and 18/20 on Square after 50 real-world episodes, compared with 10/20 and 13/20 for SimDist.
  • The abstract, introduction and limitations are updated.

2. Controlling for encoder transfer (new App. D.4, Fig. 12; Reviewer JSYh W3)

  • IBRL additionally receives a frozen source-pretrained encoder in three variants (world-model encoder, shared RL encoder, separate RL encoders).
  • RL-trained encoders reduce the initial drop of IBRL but do not raise its success at the 50k budget, and all variants remain below Dyna-IBRL at that budget.
  • Sec. 4.1 now points to this control.

3. Larger and different dynamics shifts (new App. D.5, Tabs. 14–16; Reviewer MhBX W1/W2/Q1/Q2/Q4, Reviewer 7kGB W1/Q1)

  • A ladder of five shift levels on Threading (Tab. 14).
  • Oracle-dynamics experiments show that the frozen source value remains sufficient when the shift changes how states are reached, and fails when it changes which states are good (rotated hole). This limit is now also stated in the limitations.
  • DomainADEPT outperforms SimDist and IBRL in all six configurations with λ=1\lambda=1 (Tab. 15). The multi-configuration study currently covers Threading; extending it to the remaining tasks is in progress.
  • A tether-strength ablation on shifts that require new behavior shows that a moderate tether does not prevent new behavior, while a strong tether hurts when source actions are systematically wrong (Tab. 16).

4. Ensemble mean under contact (App. D.2, Tab. 12; Reviewer MhBX Q3)

  • Per-contact-phase analysis of ensemble disagreement and 5-step prediction.
  • Disagreement during contact is comparable to free motion, and the ensemble mean is more accurate than the best single head chosen in hindsight in every seed and phase.

5. Demonstration quantity and quality (new App. D.6, Fig. 13; Reviewer JSYh Q1)

  • Results with 10 and 5 demonstrations and with non-expert demonstrations.
  • Sec. 4 now states that the 20 demonstrations (about 2k transitions) are the interaction budget.

6. Comparison with alternative adaptation strategies (new App. D.7, Fig. 14, App. C.4; Reviewer MhBX Q5)

  • Comparison with latent adaptation [1] and DARC [2] on Threading, with implementation details in App. C.4.
  • A discussion of when adapting the world model is preferable to adapting the policy or its latent input.
  • Sec. 2.1 now discusses latent adaptation [1] alongside the other approaches that use target data.

7. Presentation (Reviewer JSYh W1)

  • Pseudocode for the two-channel prior-preference margin is added as Alg. 2 (App. C.1) and referenced in Sec. 4.1.
  • The typos pointed out by the reviewer are corrected, including "25 percentage points" in the introduction.

We hope the revision addresses the concerns raised, and we are happy to answer any further questions.

References:

[1] Da, Longchao, et al. "Latent adaptation of foundation policies for sim-to-real transfer." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Eysenbach, Benjamin, et al. "Off-dynamics reinforcement learning: Training for transfer with domain classifiers." arXiv preprint arXiv:2006.13916 (2020).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.