- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper addresses the reuse of training interventions in embodied reinforcement learning by transferring trainer-side knowledge about the contexts in which specific interventions have previously helped or harmed learning. For each intervention, the proposed Training Contracts framework collects paired training comparisons and computes supported and contradicted regions. Then the selected regions are evaluated on independent source checkpoints. During training on target tasks, the trainer uses the training contracts to decide whether to reuse an intervention, conduct a budgeted paired probe, or abstain from applying it. The paper evaluates the method on twelve target configurations across six manipulation task families, with additional experiments involving Meta-World and a physical SO-101 robot. Training Contracts reports an average anytime score of 0.640, compared with 0.590 for the uncertainty-aware conditional-effect baseline CE-U. The corresponding scores are 0.615 and 0.563.
Critical strengths
-
The paper studies a useful problem. The same training intervention may help in some situations but hurt in others. Learning when an intervention should or should not be reused makes sense.
-
The paper compares with several baselines and includes ablation studies, Meta-World, and real robot experiments. The results are generally good. Training Contracts improves the anytime score from 0.590 to 0.640 compared with CE-U. It also accepts fewer harmful interventions (25 vs. 49) at the same 45% coverage.
Critical weaknesses
-
The novelty of the method is not very clear. The method combines paired comparisons, rule learning, independent confirmation, and decisions about whether to reuse an intervention. These ideas are already quite familiar. Table 2 suggests that most of the improvement comes from the learned regions. It would be helpful to explain what the other parts of Training Contracts add.
-
The experiments do not clearly support some parts of the discovery objective. Removing the rule complexity penalty () improves the anytime score from 0.64 to 0.66. Also, gives 0.65, compared with 0.64 for the default . The paper does not show whether the complexity penalty actually produces simpler rules, or whether a larger reduces harmful reuse decisions.
Questions for authors
Q1. Contribution of the full method. In Table 2, CE-U achieves 0.637 when given the learned validity regions, compared with 0.640 for Training Contracts. What additional benefit does the full method provide beyond these learned regions?
Q2. Discovery objective. Removing the rule complexity penalty improves performance from 0.64 to 0.66, and using a smaller also gives a better result. Could the authors explain why these penalties are needed?
Q3. Partially confirmed contracts. What happens if only the supported or contradicted region passes independent confirmation? Does the method keep the confirmed region and discard the other, or reject the whole contract?