- Rating
- 2
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether safe behavior from one embodiment can improve another embodiment’s policy in offline safe reinforcement learning. It proposes Cross-Embodiment Corrective Supervision (CECS), which retrieves context-matched safe source motion, translates it into target-native actions through a target-specific inverse dynamics model, filters corrections using local data support and a target cost critic, and uses the accepted labels to refine the target policy. Experiments across four simulated morphologies and two task families demonstrate improved safety and competitive safety–reward trade-offs against offline safe RL baselines in pairwise and multi-source settings. Matched-training controls show benefits beyond target-only retrieval at identical contexts and correction counts, while shuffled-label comparisons establish the importance of context matching. Matched-state rollouts further show sustained cost reductions while retaining most task progress. The validation is limited to simulation with shared task-defined motion interfaces and provides empirical safety improvements rather than formal guarantees.
Critical strengths
- The paper investigates an insightful question: can embodiments facing similar safety risks benefit from one another’s corrective behavior despite differences in morphology and dynamics? This question is consequential for offline safe RL, where a target embodiment may have limited examples of useful safety corrections in its own dataset.
- The evaluation is extensive within the chosen simulated benchmarks. It covers four morphologies, two task families, pairwise and multi-source transfer, and comparisons with offline safe RL baselines. The matched-training controls, shuffled-label comparisons, and matched-state rollouts are particularly useful for assessing whether source corrections provide value beyond additional target-only training.
Critical weaknesses
- The methodological advance relative to the complexity of the pipeline needs clearer justification. At a high level, the method retrieves source motion, translates it into target-native actions, and uses those actions as corrective supervision. The predictive variant additionally trains a shared encoder with prediction, reconstruction, and alignment objectives; the target-side pipeline also requires inverse-dynamics models before policy refinement. While the experiments support the usefulness of source corrections, the paper should explain more clearly what constitutes the technical advance in this combination and whether the added preparation stages are justified by their benefits relative to the simpler explicit-retrieval variant.
- Sensitivity to the loss weights is insufficiently evaluated. The predictive-retrieval objective in Eq. (3) combines future prediction, reconstruction, and alignment, while Eq. (9) combines native supervision, confidence-weighted corrective supervision, and entropy regularization. The paper reports settings and normalization procedures, but I could not find an empirical sensitivity analysis for the reconstruction and alignment weights or the relative strength of corrective supervision. Such an analysis would help assess robustness across source–target pairs and the manual tuning required to apply the method to a new setting.
Minor Comments
- The AI Use Statement describes assistance with drafting and editing the manuscript. If generative AI was also used to create or edit Figure 1, please clarify that use in the disclosure.
- Figure 1 is embedded as a raster image rather than a vector graphic. Please export the diagram in vector format to keep its text, lines, and arrows sharp when zoomed or printed.
Questions for authors
- What is the computational overhead of each preparation and training stage: predictive-encoder training, the five-fold target inverse-dynamics models, target cost-critic training, retrieval and correction construction, and policy pretraining and refinement? Please report hardware, elapsed time, and whether each component must be retrained for a new source–target pair, and distinguish the costs of explicit and predictive retrieval.
- [Bonus] The experiments cover distinct morphologies, but all transfer occurs within Run or Circle using shared planar motion coordinates. How far can the method generalize to robotic arms with different kinematic structures, action dimensions, and feasible workspaces? In particular, what shared motion interface and target-data coverage would be needed for the inverse model to translate source corrections reliably, and when should the method abstain because the source motion is infeasible for the target?