- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Robots must be able to assess whether the task they are supposed to execute is feasible. This judgment may change over time, as the robot wears down, links might bend, or parts might be replaced. The authors propose a method to classify tasks as feasible, infeasible, or uncertifiable. The last category may, for instance, be necessary if execution is hindered by a safety envelope.
Critical strengths
-
The problem itself seems important. The authors consider an important problem setting, which could make this a valuable contribution.
-
The authors compare their method to ten baseline methods, thus providing substantive evidence.
Critical weaknesses
-
Overall, I found the paper quite difficult to follow. The paper mentions many concepts but doesn't properly introduce some of them. Each concept has its own notation, which the reader must recall throughout the paper. For instance, it's unclear what the "witnesses" are or what "explicit shape assumptions" they add. The constant forward-referencing is more confusing than helpful. I didn't find a clear problem formulation in the corresponding section. At the high level, it is clear what the paper wants to do, but the details remain somewhat vague. Overall, this makes the paper difficult to access and limits its potential.
-
A proper literature review is missing. The connections to SafeOpt are also unclear to me; SafeOpt seems to address a very different problem setting. This makes it hard to position the paper and evaluate its relevance.
-
The assumptions underlying the theoretical statements are not clear. What does it mean for observations to be bounded? To my knowledge, the empirical Bernstein mixture supermartingale used requires several assumptions. For example, bounded random variables, a martingale difference sequence, finite variance, mixing conditions, etc. I didn't see those assumptions explicitly verified. The authors didn't make any explicit assumptions about the noise level, other than that it has zero expected value. This makes it difficult to verify the theoretical results.
Questions for authors
-
Could you clarify what assumptions you make about the system noise?
-
Can you position your paper with respect to the literature?
-
What exactly is the problem setting? Trying to classify feasible tasks from limited probing? Which would force us to also extrapolate beyond what has been tried?