- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
It explores the limitations of severity-based sample weighting in control learning, particularly when training data contains corrupted action labels. The paper shows that commonly used difficulty measures, such as action jerk and path length, can incorrectly assign higher importance to corrupted samples. To address this, it explores a learned residual weighting model and introduces a consistency-aware target that considers the agreement between actions and observed trajectories. The approach is evaluated on USV control and Push-T manipulation tasks, with comparisons against uniform weighting, Meta-Weight-Net, L2RW, and several controlled variants. The experiments show that severity-based weighting can favour corrupted samples, while consistency-aware weighting helps correct this behaviour. Additional ablations examine the effects of weighting targets, input features, and weight magnitudes. However, the presentation and writing require significant improvement, and the corrected weighting does not consistently outperform uniform training in closed-loop task performance.
Critical strengths
-
It highlights an important limitation of severity-based weighting, where corrupted action labels can be mistaken for difficult training examples and assigned higher importance.
-
It examines the effects of weighting targets, model features, and weight magnitude through controlled experiments. The comparison between genuinely difficult maneuvers and corrupted examples is particularly useful, as it shows that severity measures alone may not distinguish valid hard examples from unreliable labels.
Critical weaknesses
-
The paper is poorly organized and difficult to follow. The motivation, proposed methodology, and experimental analysis are not presented clearly, making it challenging to understand the main contributions and how the different components fit together.
-
The distinction between difficult samples and corrupted supervision has already been explored in noisy-label learning and imperfect demonstration learning. The paper does not adequately differentiate its contribution from this existing literature.
Questions for authors
-
The paper is difficult to follow, particularly the problem definition, motivation, research gap, and the description of the weighting mechanism and experimental setup.
-
Several closely related studies on imperfect demonstrations and data quality appear to be missing: (1) Wang et al., Learning to Weight Imperfect Demonstrations (ICML 2021), (2) Belkhale et al., Data Quality in Imitation Learning (NeurIPS 2023), and (3) Yue et al., How to Leverage Diverse Demonstrations in Offline Imitation Learning (ICML 2024). Could you clarify how their findings differ from these studies, and what genuinely new insight is provided beyond the known distinction between difficult and unreliable training examples?
-
The consistency-aware target reduces the importance assigned to corrupted samples; the reported closed-loop performance does not consistently outperform uniform weighting. What practical advantage does the proposed approach offer over uniform weights? Do the existing experiments provide convincing evidence that the correction consistently improves control performance?
-
The conclusion is largely based on a specific severity definition and limited corruption settings. How can you establish that the observed failure is a broader problem with severity-based weighting rather than a consequence of particular target formulation? Do the reported results provide sufficient evidence across different corruption types, tasks, and random seeds?
The paper has been revised substantially, including the framing, organisation, experimental evidence, and interpretation of its central results. I appreciate the authors’ significant efforts to do this, but I believe this calls for a fresh review, which is outside the scope of this rebuttal. It also indicates that the original version of the paper, with reasonable (minor) modifications, is not suitable for publication.
The authors also admit that many of the central claims of the previous version are not sufficiently justified and reframe the paper, changing its central claims. This includes: a) their proposed severity target is not an established weighting method; it is unclear whether this corresponds to difficulty; b) neural network training of the weights is redundant.
Some further questions remain:
a) Corrupted labels are difficult to fit: Isn’t it obvious that prediction errors (difficulty) would be high on corrupted data? This is already well established and motivates a lot of literature on data curation/cleaning. What new insight do these experiments establish beyond demonstrating this known failure mode in Push-T?
b) Reward-free intrinsic-difficulty setting: Difficulty is not clearly defined, and the setting itself is not well motivated. For example, task difficulty requires rewards to be well defined. Prediction difficulty does not. If task success rate is the objective, why can’t we use rewards to come up with the weighting?
c) Limited generalisation: The paper’s main claims rest on experiments in a single environment; how do we know that the results are not specific to the environment/dataset?
For all the above reasons, I maintain my score.