acceptodds
Under review as a conference paper at ICLR 2027

Temporally Directed Gromov–Wasserstein Imitation Learning

Abstract

Cross-domain imitation learning studies how to transfer expert behavior between agents with different embodiments. Gromov–Wasserstein matching enables comparison across different state-action spaces without learning an explicit mapping, but its global coupling makes each imitation reward depend on the complete trajectory and ignores temporal ordering. We propose Temporally Directed Gromov–Wasserstein Imitation Learning (GWτdIL), which replaces full pairwise geometry with temporally ordered comparisons of local transition magnitude and directional variation. This produces per-transition imitation rewards from a single expert demonstration while preserving cross-domain structural comparison. For equal-length trajectories with uniform weights, the resulting empirical objective can be computed in linear time. We establish sufficient conditions for policy-optimality preservation and show that task and embodiment matching impose complementary constraints on the solution space. Experiments across domains with different state dimensions and morphologies show that GWτdIL consistently learns target-directed behavior and outperforms prior cross-domain imitation methods under comparable assumptions. Ablations isolate the roles of temporal ordering, directional information, and task–embodiment matching.

Primary area:
reinforcement learning
Keywords:
Cross-Domain Imitation Learning, Robot Learning, Gromov-Wasserstein Distance, Optimal Transport, Continuous Control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_p2YwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes GW$\tau$dIL for cross-domain imitation from a single state-only demonstration. GWIL's full pairwise Gromov–Wasserstein matching is replaced by a comparison of cumulative path lengths from a root point. The path length combines the step norm ∥Δz∥\|\Delta z\| and ∣1TΔz∣|1^T\Delta z|. For equal-length, uniformly weighted trajectories, this reduces to a 1D Wasserstein distance computable in O(T). The authors derive a per-step imitation reward with a task and embodiment decomposition and evaluate it on Pendulum-CartPole and six Hopper, Walker2d, HalfCheetah transfers against GWIL and UDIL.

Critical strengths

The method is simple and cheap: O(T) reward computation is faster than in GWIL baseline.

Pendulum->CartPole swing-up reaches about 600 return using imitation reward only. GWIL fails in this IL-only setting, so this is a meaningful result.

The ablations are informative: directional term (Fig. 4), task and embodiment weighting (Fig. 5–6), and grouped GWIL (App. C.4).

Critical weaknesses

W1. Theorem 1 does not establish optimality preservation.

Consider a line MDP with z=(s,a)z = (s, a), s∈Zs \in \mathbb{Z}, a∈{−1,+1}a \in \{-1, +1\}, s′=s+as' = s + a, R(s,+1)=1R(s, +1) = 1, R(s,−1)=0R(s, -1) = 0, and take ϕ=ψ=id\phi = \psi = \mathrm{id}. Both always +1+1 and always −1-1 satisfy

Δz=(±1,0),∥Δz∥=∣1⊤Δz∣=1at every step,\Delta z = (\pm 1, 0), \qquad \|\Delta z\| = |\mathbf{1}^\top \Delta z| = 1 \quad \text{at every step},

so GWτd=0GW_{\tau d} = 0 and the worst policy is a minimizer. By Def. 2 it is even temporally directed isomorphic to the optimal policy via the reflection (s0+k,+1)↦(s0−k,−1)(s_0 + k, +1) \mapsto (s_0 - k, -1). The conclusion of Theorem 1 is therefore too weak to imply anything about return, and the directional term fails to break even this simplest reflection. Separately, step (d) infers isomorphism from zero distance by citing Proposition 2, which gives only a pseudo-metric, so this step needs its own argument. Def. 2 (Eq. 6) and step (c) also apply dτdd^{\tau d} to arbitrary pairs (z,z′)(z, z'), whereas Eq. 4 defines it only cumulatively along a rollout, as dτd(r,zi)d^{\tau d}(r, z_i). Finally, the theorem covers none of the experiments: no bijective isometry exists between spaces of different dimension (33 vs 55, 1111 vs 1717), and it is stated for state–action pairs with discounted occupancy ρπ\rho_\pi, while the method uses states only, a single rollout and uniform weights.

W2. The local reward is a heuristic, not a consequence of the objective. Since cumulative lengths are monotone, the optimal coupling is the identity t to t. The method amounts to time-aligned matching of a scalar profile. The global objective penalizes cumulative errors, whereas Eq. 13 penalizes increments.

W3. Experiments. Normalized by the target expert, final performance ranges from about 73% (Hopper-Walker) down to 3-5% (HalfCheetah targets), so "learns optimal behaviors" is overstated. The λ\lambda used in Fig. 3 and how it was selected are not stated. Steps are aligned by index across domains with different control timesteps (8 s vs 50 s per episode). There is no native return, no SAC upper bound baseline. The reimplemented UDIL is far below its original report. D3IL (Choi et al., 2023) evaluates on the same Pendulum/CartPole and Walker/Hopper/Cheetah transfers. It is cited but neither compared against the proposed method.

W4. Formal issues in Eqs. 14-15 and Theorem 2.

  • In Eq. 14, the coupling \theta does not appear in the summand, the same index t ranges over both trajectories, and one-step rather than cumulative distances are used. So Eq. 14 is not an instance of Def. 1.
  • Eq. 15 as written rewards embodiment mismatch.
  • DTD_T and DJD_J in Theorem 2 are undefined.
  • The second part of Theorem 2 is a tautology, and the claim that neither condition implies the other is not supported by an example.
Questions for authors
  1. The coupling is the identity t to t. What does the GW formulation add beyond time-aligned matching of a scalar profile?

  2. Under what conditions do the optima of Eq. 13/15 and of the cumulative objective coincide when exact matching is infeasible?

  3. Is t included in the observation? Are terminations enabled for Hopper and Walker?

  4. Which λ\lambda is used in Fig. 3 and how it was selected?

  5. Can you report results normalized by the target-domain expert and native returns, with SAC upper bounds, and a comparison with D3IL on its tasks?

6 comments
AuthorsOpenReview ↗

Response to Reviewer p2Yw [1/3]

We thank the reviewer for recognizing the computational efficiency of our method, the meaningful imitation-only results, and the informative ablations. We appreciate the opportunity to clarify these points.

We first answer the five questions, then address the remaining concerns and summarize the revisions.

Q1. The coupling is the identity t to t. What does the GW formulation add beyond time-aligned matching of a scalar profile?

The GW formulation preserves cross-domain structure. As shown in Figure 1, instead of computing structural matches between points far apart in time (e.g., t and t+100), we compare local structures around adjacent steps (e.g., t and t+1). These structures capture the magnitude of state increments and their relative directions within each domain.

We fix a point pair in the GW coupling as the root and reduce the resulting optimization to an optimal transport problem from t to t. GWIL also uses Euclidean distances to compare state-increment magnitudes across trajectory-wide point pairs and provide a scalar reward at each time step. Building on GWIL, we retain the most effective structural information while reducing computation.

Q2. Under what conditions do the optima of Eq. 13/15 and of the cumulative objective coincide when exact matching is infeasible?

When exact matching is infeasible, the two objectives are not mathematically guaranteed to have the same optimizer and the local objective serves as a surrogate for the cumulative objective. Our experiments show that when exact matching is infeasible, the local objective outperforms the cumulative objective, further supporting the need for local structural rewards.

Reward comparison beyond exact matching. Results report native return and signed displacement after two million environment steps, as means ± standard deviations across the three training seeds used for this comparison.

MethodNative returnTravelling distance (m)
GWτdIL local reward2050.99 ± 85.5416.16 ± 0.62
Cumulative Objective Reward914.92 ± 11.756.26 ± 0.55
GWIL166.41 ± 92.821.08 ± 0.77

The local reward achieves higher native return and travelling distance than both baselines, supporting its practical effectiveness beyond exact matching. These results are consistent with our motivation for moving from global comparisons to local structural rewards, which provide simpler and more direct feedback for policy learning.

Q3. Is t included in the observation? Are terminations enabled for Hopper and Walker?

Yes. The time index augments the policy observation but is excluded from the structural-distance computation.

No. For Hopper and Walker, termination on unhealthy states is disabled during both training and evaluation; episodes are truncated at 1,000 steps. Learners in the imitation-only experiments receive only imitation rewards, without an additional termination signal from unhealthy states.

Q4. Which λ is used in Fig. 3 and how was it selected?

Figure 3 uses λ=0.5\lambda=0.5 for all six transfers, chosen to weight task and embodiment matching equally without task-specific tuning. We have stated this explicitly in Section 5.2. The weighting ablations examine the effects of other values.

AuthorsOpenReview ↗

Response to Reviewer p2Yw [2/3]

Q5. Can you report results normalized by the target-domain expert and native returns, with SAC upper bounds, and a comparison with D3IL on its tasks?

1. Target-expert normalization and native returns. The original experiments used different control intervals (HalfCheetah: 0.05 s; Hopper/Walker: 0.008 s), complicating direct displacement comparisons. We added experiments with all control intervals set to 0.008 s and report displacement and native return in Appendix C.5 to make the displacement comparisons consistent in simulated duration.

Transfer performance with matched control intervals. The table reports means ± standard deviations across 3 training seeds. The last column divides learner displacement by the target-domain expert demonstration’s displacement (SAC reference upper bounds): 20.46 m for Hopper, 32.82 m for Walker, and 35.95 m for HalfCheetah008 (Table B.1).

TransferGWIL Travelling distanceGWIL Native returnGWIL Source-expert displacement ratioGWτdIL Travelling distanceGWtdIL Native returnGWτdIL Source-expert displacement ratio
Walker → HalfCheetah0080.95 ± 0.60 m−118.53 ± 53.772.89% ± 1.83%30.92 ± 0.65 m3525.18 ± 86.0894.22% ± 1.97%
Hopper → HalfCheetah0080.61 ± 0.07 m−171.34 ± 25.392.99% ± 0.34%21.98 ± 0.34 m2406.41 ± 22.15107.43% ± 1.66%

Across these transfers under matched control intervals, GWτdIL achieves strong absolute performance and clearly outperforms GWIL (Figures C.4–C.5). These results demonstrate that our temporally ordered local geometric comparison improves upon GWIL’s global matching, yielding greater travelling distances.

2. Comparison with D3IL. Although D3IL uses a different information setting, we agree that including this strong baseline strengthens the evaluation. D3IL learns representations from visual observations and additional non-expert data, whereas GWτdIL constructs rewards directly from a single state-based demonstration.

Performance comparison with D3IL. Results are means ± standard deviations across three training seeds. We report native return for CartPole and travelling distance for the locomotion tasks (Appendix C.6).

TransferGWτdILD3IL
Pendulum → CartPole732.27 ± 27.67635.99 ± 65.49
Walker → Hopper16.16 ± 0.62 m−13.01 ± 9.15 m
Walker → HalfCheetah00830.92 ± 0.65 m−4.27 ± 11.10 m

GWτdIL achieves higher final mean performance on all three tasks without a separate representation-learning stage. D3IL uses an additional 50,000 representation pretraining updates in our experiments.

We address below the remaining weaknesses not covered by our responses to the questions.

W1. Theorem 1 does not establish optimality preservation.

1. Corrected directional term. Our implementation uses cD,k=1D⊤(zD,k+1−zD,k)∥1D∥2∥zD,k+1−zD,k∥2∈[−1,1]c_{D,k}=\frac{\mathbf{1}_{D}^{\top}(\boldsymbol{z}_{D,k+1}-\boldsymbol{z}_{D,k})}{\lVert\mathbf{1}_{D}\rVert_{2}\lVert\boldsymbol{z}_{D,k+1}-\boldsymbol{z}_{D,k}\rVert_{2}}\in[-1,1] with directional component (1+cD,k)/2(1+c_{D,k})/2,which has been anonymously released on September 26, 2026. All reported GWτdIL experiments with the directional term enabled used this formulation. In your example, increments (1,0)(1,0) and (−1,0)(-1,0) give cD,k=1/2c_{D,k}=1/\sqrt{2} and −1/2-1/\sqrt{2}, respectively. Their directional components differ, so the two rollouts no longer have zero structural distance. We have corrected Definition 1 and Eq. (4) to match the implementation and apologize for the confusion.

2. Direct isomorphism proof and explicit pairwise definition. We have added an independent argument in Appendix A.3(d). For equal-length, uniformly weighted rollouts, Proposition 1 and zero cost imply equal cumulative vectors at every index, hence equal pairwise structural differences. The indexwise bijection preserves roots, pairwise structure, and empirical measures, directly proving isomorphism in Appendix A.3(d). The pairwise definitions added after Eq. (4), dDτd(zD,i,zD,j):=∣dDτd(rD,zD,i)−dDτd(rD,zD,j)∣\boldsymbol{d}_{D}^{\tau d}(z_{D,i},z_{D,j}):=\left|\boldsymbol{d}_{D}^{\tau d}(r_{D},z_{D,i})-\boldsymbol{d}_{D}^{\tau d}(r_{D},z_{D,j})\right| and dDτd(zD,i,zD,j):=∥dDτd(zD,i,zD,j)∥2d_{D}^{\tau d}(z_{D,i},z_{D,j}):=\left\lVert\boldsymbol{d}_{D}^{\tau d}(z_{D,i},z_{D,j})\right\rVert_{2}, support Definition 2 and steps (c)–(d).

AuthorsOpenReview ↗

Response to Reviewer p2Yw [3/3]

3. Experiments for Theorem 1

  • We evaluate mirrored PointMass Maze transfer in Appendix C.7 to examine whether GWτdIL transfers the structure of an optimal expert policy. The setting satisfies Theorem 1’s assumptions and uses state–action pairs. Theorem 1 therefore predicts that a GWτdIL policy minimizing the defined objective has specified rollouts that are temporally directed isomorphic to an optimal target policy. By looking at the geometry of the maze, the learner closely reproduces the expert’s path structure, providing support for Theorem 1.

  • For state-only inputs, the construction of πA⋆\pi_A^\star remains valid, and steps (c)–(d) apply with zD=sDz_D=s_D under the corresponding structural-preservation assumptions. The guarantee is about trajectory isomorphism and actions are learned through interaction with the learner’s dynamics. Following GWIL, we construct metric measure spaces using a single demonstration and uniform empirical measures, keeping demonstration requirements minimal.

2. GWτdIL’s Optimality Scope and Learning Acceleration. Cross-domain imitation between substantially different morphologies is generally under-defined (Franzmeyer et al., 2022): its goal is to learn a policy resembling the expert’s, whereas native-reward optimization seeks to maximize target-domain return. Without assumptions linking the domains and their objectives, structural imitation does not generally guarantee maximum target-domain return. Despite this limitation, GWτdIL’s simple formulation enables rapid learning from an accessible demonstration. We compare SAC trained with environment rewards, imitation rewards, and their combination under the same health-based termination setting (Appendix C.10).

Reward comparison using SAC. Results report travelling distance at four training checkpoints, as means ± standard deviations across three training seeds.

Training reward0.5M steps Distance (m)1M steps Distance (m)1.5M steps Distance (m)2M steps Distance (m)
Environment reward4.35 ± 1.527.47 ± 1.7114.01 ± 1.1718.25 ± 1.56
GWτdIL8.92 ± 0.2511.51 ± 0.1413.98 ± 0.9716.16 ± 0.62
GWτdIL + environment reward11.50 ± 1.7516.68 ± 4.5519.30 ± 4.8023.07 ± 1.79

GWτdIL alone yields higher mean displacement early in training than environment reward alone. Combining the rewards yields the highest mean displacement at every checkpoint, showing that GWτdIL guides early learning and complements environment-reward optimization.

W4. Formal issues in Eqs. (14)–(15) and Theorem 2.

We explicitly define DT(πA)=GWτd,T2(πE,πA)D_{T}(\pi_{A})=GW_{\tau d,T}^{2}(\pi_{E},\pi_{A}) and DJ(πA)=GWτd,J2(πE,πA)D_{J}(\pi_{A})=GW_{\tau d,J}^{2}(\pi_{E},\pi_{A}), with their expressions in Section 4.2, and correct the placement of the square in Eq. (15). The original Eq. (14) used one-step discrepancies and was therefore not a direct instance of Definition 1. We have revised Section 4.2 to distinguish the cumulative structural objectives from their local reward surrogate.We also streamline Theorem 2 and add counterexamples.

UDIL implementation. We use a shared evaluation protocol for all methods. With no executable code in the authors’ linked repository, we reimplemented UDIL using the published method and reported hyperparameters (Appendix B.3). Cross-domain imitation performance also depends on expert demonstrations, so it is reasonable that our reproduction does not exactly match the results reported in the original UDIL paper.

Reviewer_ndTQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a cross-domain imitation learning method that learns from a single expert demonstration of a different embodiment. Instead of GWIL’s global Gromov–Wasserstein coupling over complete trajectories, GWτdIL compares temporally ordered, per-transition structural quantities (transition magnitude plus a directional term) between expert and learner. This yields per-step imitation rewards computable in linear time for equal-length trajectories. The state is split into task and embodiment components with a weighting parameter. Theory covers optimality preservation under isometric MDPs and the complementary constraints of task and embodiment matching. Experiments on Pendulum-to-CartPole and six directed transfers among Hopper, Walker2d, and HalfCheetah, trained with SAC, compare against GWIL and UDIL, with ablations on direction, temporal order, and the task/embodiment weighting.

Critical strengths
  • The method is simple and fast to compute, and it requires only a single expert demonstration.
  • The ablation studies are fairly comprehensive, covering direction, timing, task/embodiment, and other factors.
Critical weaknesses
  • The method essentially matches a scalar step size at each time step, so the Gromov-Wasserstein (GW) framing is largely cosmetic. The “directional term” sums over coordinates, which depends on the coordinate system and sign convention, and it is not invariant under the isometries that GW is designed around. Equation (4) also takes the absolute value of this term, so moving forward and moving backward give the same value. Where, then, does the directional information come from? This is internally inconsistent.

  • Embodiments differ in their natural step scales. For example, an expert HalfCheetah runs 655 m over 1,000 steps, while Hopper runs only 20 m. Yet the method matches raw magnitudes without normalization. Meanwhile, HalfCheetah-from-Hopper reaches only about 25 m, and HalfCheetah-from-Walker about 35 m, far below what HalfCheetah itself can achieve. The paper reports only displacement, not native returns.

Questions for authors

Please see the weaknesses above.

I would be happy to raise my score if they are resolved.

8 comments
AuthorsOpenReview ↗

Response to Reviewer ndTQ [1/2]

We thank the reviewer for recognizing the simplicity and computational efficiency of our method and the coverage of our ablations. We address the concerns about the structural formulation, directional information, and experimental evaluation below.

The method essentially matches a scalar step size at each time step, so the Gromov-Wasserstein (GW) framing is largely cosmetic.

1. Structural information beyond scalar step size. Although the policy is trained with a scalar reward, GWτdIL separately represents movement magnitude and relative direction. The cumulative structural representation is dDτd(rD,zD,i)=∑k=1i−1(∥zD,k+1−zD,k∥2,1+cD,k2)⊤d_{D}^{\tau d}(r_{D},z_{D,i})=\sum_{k=1}^{i-1}\left(\lVert z_{D,k+1}-z_{D,k}\rVert_{2},\frac{1+c_{D,k}}{2}\right)^{\top}, and its temporal alignment follows from its optimal-transport construction.

2. Optimal temporal correspondence and computational efficiency. Our construction follows the GW principle of comparing within-domain relations across domains whose coordinates may have different dimensions. We replace the full pairwise GW objective with optimal transport between cumulative structural representations. Because both components are nondecreasing, Proposition 1 establishes that time-index matching is optimal for equal-length, uniformly weighted trajectories. This yields O(T)O(T) computation for fixed state dimensions. The contribution is therefore a structural representation and a provably optimal temporal correspondence that support efficient local imitation rewards (Section 4.1, Proposition 1; Section 4.2, Remark 2).

The “directional term” sums over coordinates, which depends on the coordinate system and sign convention, and it is not invariant under the isometries that GW is designed around. Equation (4) also takes the absolute value of this term, so moving forward and moving backward give the same value. Where, then, does the directional information come from? This is internally inconsistent.

1.Corrected formulation consistent with the released implementation. We apologize for the confusion caused by the original formulation. We have corrected Definition 1 and Eq. (4) to match the directional term actually used in our implementation, anonymously released on September 26, 2026. For nonzero increments, the implementation uses the signed cosine similarity cD,k=1D⊤(zD,k+1−zD,k)∥1D∥2∥zD,k+1−zD,k∥2c_{D,k}=\frac{1_{D}^{\top}(z_{D,k+1}-z_{D,k})}{\lVert 1_{D}\rVert_{2}\lVert z_{D,k+1}-z_{D,k}\rVert_{2}}, with directional component (1+cD,k)/2(1+c_{D,k})/2. All reported GWτdIL experiments with the directional term enabled used this formulation. Reversing an increment changes cD,kc_{D,k} to −cD,k-c_{D,k}, distinguishing opposite motions when the projection is nonzero. For example, increments +1+1 and −1-1 yield structural increments (1,1)⊤(1,1)^\top and (1,0)⊤(1,0)^\top, respectively.

2.Reference-dependent direction and the scope of Theorem 1. We deliberately retain direction relative to a chosen reference, allowing equal-magnitude motions with opposite projections to be distinguished. This directional sensitivity is useful for imitation tasks where motion orientation matters. In our cross-domain comparison it requires consistent directional references. Theorem 1 applies to mappings that preserve both structural components, rather than arbitrary isometries.

AuthorsOpenReview ↗

Response to Reviewer ndTQ [2/2]

Embodiments differ in their natural step scales. For example, an expert HalfCheetah runs 655 m over 1,000 steps, while Hopper runs only 20 m. Yet the method matches raw magnitudes without normalization. Meanwhile, HalfCheetah-from-Hopper reaches only about 25 m, and HalfCheetah-from-Walker about 35 m, far below what HalfCheetah itself can achieve. The paper reports only displacement, not native returns.

We address this concern through implementation details and additional experiments with matched control intervals.

1. Magnitude normalization in the implementation. In the released implementation, each compared pair of expert and learner increments is divided by a common maximum absolute coordinate value, and the squared magnitude discrepancy is normalized by the larger feature dimension. This controls the magnitude-loss scale while retaining relative magnitude differences.

2. Matched control intervals and native-return evaluation. The original demonstrations also used different control intervals (0.05 s for HalfCheetah and 0.008 s for Hopper/Walker), complicating direct displacement comparisons. Consequently, 1,000 steps represented 50 s and 8 s, respectively. We added experiments with all control intervals set to 0.008 s and report travelling distance and native return for all six transfers in Appendix C.5. Native returns include forward-velocity rewards and control costs, with an additional healthy reward for Hopper and Walker. Across the three environments, native-return results are broadly consistent with travelling-distance results.

HalfCheetah transfer with matched control intervals. The table reports final performance after two million environment steps, as means ± standard deviations across the three training seeds used in Figures C.4–C.5. The source-expert displacement ratio uses 20.46 m for Hopper and 32.82 m for Walker.

TransferNative returnTravelling distance (m)Source-expert displacement ratio
Hopper → HalfCheetah0082406.41 ± 22.1521.98 ± 0.34107.41% ± 1.64%
Walker → HalfCheetah0083525.18 ± 86.0830.92 ± 0.6594.22% ± 1.97%

Under matched intervals, these learners reproduce approximately the displacement of their source demonstrations. Relative to the target-domain expert’s 35.95 m, they achieve 61.1% and 86.0%, respectively. Morphology-dependent differences remain, but GWτdIL retains its performance advantage over GWIL and UDIL across all six transfers, with native returns supporting the displacement results (Figures C.4–C.5).

We hope these clarifications and additional experiments address your concerns and we would be happy to answer any further questions.

Thank you for the detailed response and the additional experiments. The clarification on control intervals is helpful, and I acknowledge that it partly explains the displacement gap I noted. Two central concerns remain:

  1. Reference direction and the GW framing. The response confirms that the full GW objective is replaced by 1D OT between monotone cumulative features, and that the directional term depends on a chosen reference that must be consistent across domains. This amounts to supplying a manual cross-domain correspondence, which GW-based methods are designed to avoid. How is the reference direction chosen in each state space, how sensitive are the results to this choice, and do GWIL and UDIL receive equivalent information? Also, since the directional component is now a signed cosine, can it be negative? If so, how does the cumulative representation remain nondecreasing, as Proposition 1 requires?

  2. Source-scale imitation. Under matched intervals, the learner reproduces the source expert’s displacement (107% / 94%) but reaches only 61% of the HalfCheetah expert when learning from Hopper. This suggests the method transfers the source embodiment’s absolute step scale rather than the task objective. The normalization in the implementation keeps relative magnitude differences, so it does not remove this effect. Could you discuss this limitation explicitly, and report the native return of the HalfCheetah008 expert as a reference?

I still believe the revisions are substantial, and I will keep my current score

Reviewer_ZWmNOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GWτdIL, which replaces GWIL’s global Gromov–Wasserstein coupling with a time-aligned comparison of how much the state changes at each step, plus a directional term. This gives a dense, linear-time imitation reward from a single state-only demonstration from another domain. Trained with SAC, the method outperforms GWIL and UDIL on Pendulum→CartPole and on six Hopper/Walker/HalfCheetah transfers, and ablations study the directional term, temporal order, and the task–embodiment weighting.

Critical strengths
  • A simple, efficient fix for the time-alignment issue in GWIL. Reducing the comparison to a time-aligned 1D profile turns GWIL’s trajectory-level reward into a per-step reward. It costs O(T) to compute and trains about 3× faster, which directly addresses the credit-assignment and instability issues that limit GWIL.
  • Consistent gains, plus useful ablations. The method beats both baselines on all seven transfers from a single demonstration. The ablations (removing the directional term, the λ sweep, and grouped GWIL in Appendix C.4) help show that the gains come from the local, ordered comparison rather than from the task–embodiment split alone.
Critical weaknesses

It’s hard to tell how much cross-domain structure is actually transferred, which is the paper’s central claim. The “task” features are the same physical quantities (torso height, root velocities) for all three robots, and absolute performance is low in most transfers: HalfCheetah reaches ~25–35 m against the 655 m expert, and Walker-from-HalfCheetah ~4 m against 33 m.

Questions for authors
  1. How does the directional term tell forward from backward motion?
  2. Is the time index t part of the policy input? Is early termination turned off for Hopper and Walker?
  3. How would a simple baseline that directly matches the shared task features (torso height, root velocities) perform?
2 comments
AuthorsOpenReview ↗

Response to Reviewer ZWmN

We thank the reviewer for recognizing our method’s efficiency and ablations. We address the questions and concerns below.

Q1. How does the directional term tell forward from backward motion?

Definition 1 and Eq. (4) (lines 174–179) now match our anonymously released implementation (September 26, 2026). The directional component is (1+cD,k)/2∈[0,1](1+c_{D,k})/2\in[0,1], where cD,k=1D⊤(zD,k+1−zD,k)∥1D∥2∥zD,k+1−zD,k∥2∈[−1,1]c_{D,k}=\frac{1_{D}^{\top}(z_{D,k+1}-z_{D,k})}{\lVert 1_{D}\rVert_{2}\lVert z_{D,k+1}-z_{D,k}\rVert_{2}}\in[-1,1] is the signed cosine between the increment and all-ones vector, yielding different directional values. For example, increments (1,0)(1,0) and (−1,0)(-1,0) give cD,k=1/2c_{D,k}=1/\sqrt{2} and −1/2-1/\sqrt{2}, respectively. All reported GWτdIL experiments with the directional term enabled used this formulation.

Q2. Is the time index t part of the policy input? Is early termination turned off for Hopper and Walker?

Yes to both questions.

  • The time index augments the policy observation but is excluded from the structural-distance computation (wrapper.py; Section 4.1, lines 155–156).
  • Health-based early termination is disabled for both Hopper and Walker during training and evaluation. Both learners are trained using imitation rewards only. The settings are specified in task_state.py and evaluation.py and clarified in Appendix B.1 (lines 869–871).

Q3. How would a simple baseline that directly matches the shared task features perform?

Direct feature matching learns forward locomotion, but GWτdIL achieves higher final performance in cross-domain IL. We added a baseline that directly matches time-aligned torso height and horizontal root velocity in HalfCheetah008→Hopper transfer (Appendix C.9, Figure C.9).

Direct feature alignment comparison. Values are means ± standard deviations across the 3 training seeds.

MethodNative returnTravelling distance
Direct Feature Alignment1051.84 ± 286.348.38 ± 2.29 m
GWτdIL (ours)1629.49 ± 182.5012.42 ± 1.72 m

Just as an infant learning to walk need not reproduce an adult’s torso height or speed, cross-domain imitation should accommodate differences in body geometry and dynamics. This motivates comparing movement structure rather than requiring direct feature values.

W1. How much cross-domain structure is actually transferred?

GWτdIL simplifies GWIL’s structural matching by replacing full pairwise comparisons with temporally ordered local comparisons of movement magnitude and projected direction. Our ablation studies show that these retained components support effective behavioral transfer. To make the resulting behavioral transfer observable, we added mirrored PointMass Maze experiments (Appendix C.7, Figure C.7 ).

Across different expert trajectory structures, the learner preserves the main geometric features of the expert trajectories: rounded demonstrations yield smoother trajectories, whereas rectilinear demonstrations yield more pronounced straight segments and turns.

W2. Absolute performance is low in several transfers.

The original experiments used different control intervals (HalfCheetah: 0.05 s; Hopper/Walker: 0.008 s), complicating direct displacement comparisons. We added experiments with all control intervals set to 0.008 s and report displacement and native return in Appendix C.5, addressing this issue.

Transfer performance with matched control intervals. The table reports means ± standard deviations across 3 training seeds.

TransferGWIL Travelling distanceGWIL Native returnGWIL Source-expert displacement ratioGWτdIL Travelling distanceGWtdIL Native returnGWτdIL Source-expert displacement ratio
Walker → HalfCheetah0080.95 ± 0.60 m−118.53 ± 53.772.89% ± 1.83%30.92 ± 0.65 m3525.18 ± 86.0894.22% ± 1.97%
Hopper → HalfCheetah0080.61 ± 0.07 m−171.34 ± 25.392.99% ± 0.34%21.98 ± 0.34 m2406.41 ± 22.15107.43% ± 1.66%

Across these transfers under matched control intervals, GWτdIL achieves strong absolute performance and clearly outperforms GWIL (Figures C.4–C.5). These results demonstrate that our temporally ordered local geometric comparison improves upon GWIL’s global matching, yielding greater travelling distances.

AuthorsOpenReview ↗

Looking Forward to Your Response

Dear reviewer ZWmN,
Thanks again for your valuable comments and suggestions, and we have done our best to answer your questions.

  1. The directional term uses signed cosine similarity to distinguish forward from backward motion.
  2. The time index t is part of the policy input but is excluded from the distance computation. Health-based early termination is disabled for Hopper and Walker.
  3. We compare direct matching of shared task features with GWτdIL, showing that GWτdIL achieves better cross-domain transfer performance.
  4. Our Maze experiments show that the learner preserves the main geometric features of the expert trajectories.
  5. After aligning the control intervals of HalfCheetah, Walker, and Hopper, GWτdIL achieves strong absolute performance on the HalfCheetah transfer tasks.

We look forward to your response and are eager to continue our discussion.

Sincerely, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.