- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper proposes GW$\tau$dIL for cross-domain imitation from a single state-only demonstration. GWIL's full pairwise Gromov–Wasserstein matching is replaced by a comparison of cumulative path lengths from a root point. The path length combines the step norm and . For equal-length, uniformly weighted trajectories, this reduces to a 1D Wasserstein distance computable in O(T). The authors derive a per-step imitation reward with a task and embodiment decomposition and evaluate it on Pendulum-CartPole and six Hopper, Walker2d, HalfCheetah transfers against GWIL and UDIL.
Critical strengths
The method is simple and cheap: O(T) reward computation is faster than in GWIL baseline.
Pendulum->CartPole swing-up reaches about 600 return using imitation reward only. GWIL fails in this IL-only setting, so this is a meaningful result.
The ablations are informative: directional term (Fig. 4), task and embodiment weighting (Fig. 5–6), and grouped GWIL (App. C.4).
Critical weaknesses
W1. Theorem 1 does not establish optimality preservation.
Consider a line MDP with , , , , , , and take . Both always and always satisfy
so and the worst policy is a minimizer. By Def. 2 it is even temporally directed isomorphic to the optimal policy via the reflection . The conclusion of Theorem 1 is therefore too weak to imply anything about return, and the directional term fails to break even this simplest reflection. Separately, step (d) infers isomorphism from zero distance by citing Proposition 2, which gives only a pseudo-metric, so this step needs its own argument. Def. 2 (Eq. 6) and step (c) also apply to arbitrary pairs , whereas Eq. 4 defines it only cumulatively along a rollout, as . Finally, the theorem covers none of the experiments: no bijective isometry exists between spaces of different dimension ( vs , vs ), and it is stated for state–action pairs with discounted occupancy , while the method uses states only, a single rollout and uniform weights.
W2. The local reward is a heuristic, not a consequence of the objective. Since cumulative lengths are monotone, the optimal coupling is the identity t to t. The method amounts to time-aligned matching of a scalar profile. The global objective penalizes cumulative errors, whereas Eq. 13 penalizes increments.
W3. Experiments. Normalized by the target expert, final performance ranges from about 73% (Hopper-Walker) down to 3-5% (HalfCheetah targets), so "learns optimal behaviors" is overstated. The used in Fig. 3 and how it was selected are not stated. Steps are aligned by index across domains with different control timesteps (8 s vs 50 s per episode). There is no native return, no SAC upper bound baseline. The reimplemented UDIL is far below its original report. D3IL (Choi et al., 2023) evaluates on the same Pendulum/CartPole and Walker/Hopper/Cheetah transfers. It is cited but neither compared against the proposed method.
W4. Formal issues in Eqs. 14-15 and Theorem 2.
- In Eq. 14, the coupling \theta does not appear in the summand, the same index t ranges over both trajectories, and one-step rather than cumulative distances are used. So Eq. 14 is not an instance of Def. 1.
- Eq. 15 as written rewards embodiment mismatch.
- and in Theorem 2 are undefined.
- The second part of Theorem 2 is a tautology, and the claim that neither condition implies the other is not supported by an example.
Questions for authors
-
The coupling is the identity t to t. What does the GW formulation add beyond time-aligned matching of a scalar profile?
-
Under what conditions do the optima of Eq. 13/15 and of the cumulative objective coincide when exact matching is infeasible?
-
Is t included in the observation? Are terminations enabled for Hopper and Walker?
-
Which is used in Fig. 3 and how it was selected?
-
Can you report results normalized by the target-domain expert and native returns, with SAC upper bounds, and a comparison with D3IL on its tasks?
6 comments
Response to Reviewer p2Yw [2/3]
Q5. Can you report results normalized by the target-domain expert and native returns, with SAC upper bounds, and a comparison with D3IL on its tasks?
1. Target-expert normalization and native returns. The original experiments used different control intervals (HalfCheetah: 0.05 s; Hopper/Walker: 0.008 s), complicating direct displacement comparisons. We added experiments with all control intervals set to 0.008 s and report displacement and native return in Appendix C.5 to make the displacement comparisons consistent in simulated duration.
Transfer performance with matched control intervals. The table reports means ± standard deviations across 3 training seeds. The last column divides learner displacement by the target-domain expert demonstration’s displacement (SAC reference upper bounds): 20.46 m for Hopper, 32.82 m for Walker, and 35.95 m for HalfCheetah008 (Table B.1).
| Transfer | GWIL Travelling distance | GWIL Native return | GWIL Source-expert displacement ratio | GWτdIL Travelling distance | GWtdIL Native return | GWτdIL Source-expert displacement ratio |
|---|---|---|---|---|---|---|
| Walker → HalfCheetah008 | 0.95 ± 0.60 m | −118.53 ± 53.77 | 2.89% ± 1.83% | 30.92 ± 0.65 m | 3525.18 ± 86.08 | 94.22% ± 1.97% |
| Hopper → HalfCheetah008 | 0.61 ± 0.07 m | −171.34 ± 25.39 | 2.99% ± 0.34% | 21.98 ± 0.34 m | 2406.41 ± 22.15 | 107.43% ± 1.66% |
Across these transfers under matched control intervals, GWτdIL achieves strong absolute performance and clearly outperforms GWIL (Figures C.4–C.5). These results demonstrate that our temporally ordered local geometric comparison improves upon GWIL’s global matching, yielding greater travelling distances.
2. Comparison with D3IL. Although D3IL uses a different information setting, we agree that including this strong baseline strengthens the evaluation. D3IL learns representations from visual observations and additional non-expert data, whereas GWτdIL constructs rewards directly from a single state-based demonstration.
Performance comparison with D3IL. Results are means ± standard deviations across three training seeds. We report native return for CartPole and travelling distance for the locomotion tasks (Appendix C.6).
| Transfer | GWτdIL | D3IL |
|---|---|---|
| Pendulum → CartPole | 732.27 ± 27.67 | 635.99 ± 65.49 |
| Walker → Hopper | 16.16 ± 0.62 m | −13.01 ± 9.15 m |
| Walker → HalfCheetah008 | 30.92 ± 0.65 m | −4.27 ± 11.10 m |
GWτdIL achieves higher final mean performance on all three tasks without a separate representation-learning stage. D3IL uses an additional 50,000 representation pretraining updates in our experiments.
We address below the remaining weaknesses not covered by our responses to the questions.
W1. Theorem 1 does not establish optimality preservation.
1. Corrected directional term. Our implementation uses with directional component ,which has been anonymously released on September 26, 2026. All reported GWτdIL experiments with the directional term enabled used this formulation. In your example, increments and give and , respectively. Their directional components differ, so the two rollouts no longer have zero structural distance. We have corrected Definition 1 and Eq. (4) to match the implementation and apologize for the confusion.
2. Direct isomorphism proof and explicit pairwise definition. We have added an independent argument in Appendix A.3(d). For equal-length, uniformly weighted rollouts, Proposition 1 and zero cost imply equal cumulative vectors at every index, hence equal pairwise structural differences. The indexwise bijection preserves roots, pairwise structure, and empirical measures, directly proving isomorphism in Appendix A.3(d). The pairwise definitions added after Eq. (4), and , support Definition 2 and steps (c)–(d).
Response to Reviewer p2Yw [3/3]
3. Experiments for Theorem 1
-
We evaluate mirrored PointMass Maze transfer in Appendix C.7 to examine whether GWτdIL transfers the structure of an optimal expert policy. The setting satisfies Theorem 1’s assumptions and uses state–action pairs. Theorem 1 therefore predicts that a GWτdIL policy minimizing the defined objective has specified rollouts that are temporally directed isomorphic to an optimal target policy. By looking at the geometry of the maze, the learner closely reproduces the expert’s path structure, providing support for Theorem 1.
-
For state-only inputs, the construction of remains valid, and steps (c)–(d) apply with under the corresponding structural-preservation assumptions. The guarantee is about trajectory isomorphism and actions are learned through interaction with the learner’s dynamics. Following GWIL, we construct metric measure spaces using a single demonstration and uniform empirical measures, keeping demonstration requirements minimal.
2. GWτdIL’s Optimality Scope and Learning Acceleration. Cross-domain imitation between substantially different morphologies is generally under-defined (Franzmeyer et al., 2022): its goal is to learn a policy resembling the expert’s, whereas native-reward optimization seeks to maximize target-domain return. Without assumptions linking the domains and their objectives, structural imitation does not generally guarantee maximum target-domain return. Despite this limitation, GWτdIL’s simple formulation enables rapid learning from an accessible demonstration. We compare SAC trained with environment rewards, imitation rewards, and their combination under the same health-based termination setting (Appendix C.10).
Reward comparison using SAC. Results report travelling distance at four training checkpoints, as means ± standard deviations across three training seeds.
| Training reward | 0.5M steps Distance (m) | 1M steps Distance (m) | 1.5M steps Distance (m) | 2M steps Distance (m) |
|---|---|---|---|---|
| Environment reward | 4.35 ± 1.52 | 7.47 ± 1.71 | 14.01 ± 1.17 | 18.25 ± 1.56 |
| GWτdIL | 8.92 ± 0.25 | 11.51 ± 0.14 | 13.98 ± 0.97 | 16.16 ± 0.62 |
| GWτdIL + environment reward | 11.50 ± 1.75 | 16.68 ± 4.55 | 19.30 ± 4.80 | 23.07 ± 1.79 |
GWτdIL alone yields higher mean displacement early in training than environment reward alone. Combining the rewards yields the highest mean displacement at every checkpoint, showing that GWτdIL guides early learning and complements environment-reward optimization.
W4. Formal issues in Eqs. (14)–(15) and Theorem 2.
We explicitly define and , with their expressions in Section 4.2, and correct the placement of the square in Eq. (15). The original Eq. (14) used one-step discrepancies and was therefore not a direct instance of Definition 1. We have revised Section 4.2 to distinguish the cumulative structural objectives from their local reward surrogate.We also streamline Theorem 2 and add counterexamples.
UDIL implementation. We use a shared evaluation protocol for all methods. With no executable code in the authors’ linked repository, we reimplemented UDIL using the published method and reported hyperparameters (Appendix B.3). Cross-domain imitation performance also depends on expert demonstrations, so it is reasonable that our reproduction does not exactly match the results reported in the original UDIL paper.
Response to Reviewer p2Yw [1/3]
We thank the reviewer for recognizing the computational efficiency of our method, the meaningful imitation-only results, and the informative ablations. We appreciate the opportunity to clarify these points.
We first answer the five questions, then address the remaining concerns and summarize the revisions.
The GW formulation preserves cross-domain structure. As shown in Figure 1, instead of computing structural matches between points far apart in time (e.g., t and t+100), we compare local structures around adjacent steps (e.g., t and t+1). These structures capture the magnitude of state increments and their relative directions within each domain.
We fix a point pair in the GW coupling as the root and reduce the resulting optimization to an optimal transport problem from t to t. GWIL also uses Euclidean distances to compare state-increment magnitudes across trajectory-wide point pairs and provide a scalar reward at each time step. Building on GWIL, we retain the most effective structural information while reducing computation.
When exact matching is infeasible, the two objectives are not mathematically guaranteed to have the same optimizer and the local objective serves as a surrogate for the cumulative objective. Our experiments show that when exact matching is infeasible, the local objective outperforms the cumulative objective, further supporting the need for local structural rewards.
Reward comparison beyond exact matching. Results report native return and signed displacement after two million environment steps, as means ± standard deviations across the three training seeds used for this comparison.
The local reward achieves higher native return and travelling distance than both baselines, supporting its practical effectiveness beyond exact matching. These results are consistent with our motivation for moving from global comparisons to local structural rewards, which provide simpler and more direct feedback for policy learning.
Yes. The time index augments the policy observation but is excluded from the structural-distance computation.
No. For Hopper and Walker, termination on unhealthy states is disabled during both training and evaluation; episodes are truncated at 1,000 steps. Learners in the imitation-only experiments receive only imitation rewards, without an additional termination signal from unhealthy states.
Figure 3 uses λ=0.5 for all six transfers, chosen to weight task and embodiment matching equally without task-specific tuning. We have stated this explicitly in Section 5.2. The weighting ablations examine the effects of other values.