- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper addresses the problem of long-horizon underwater control where compute is limited and it needs to decide when to trigger and allocate compute to different modules (state refiner, safety verifier, and a capability planner). To do this, the authors present matched-state counterfactual supervision to learn a router that at runtime, will determine which modules to use. This is done, at simulation, at the same initial state roll out what using different modules would do, to generate labels. The authors then tested in a simulator across different tasks (with labels learned in the same tasks).
Critical strengths
- Strengths
- The approach seems well-motivated and sound, where users may want to learn how to best route and does so by learning from simulator-based rollouts of picking different modules from the same initial state.
- Interesting application of underwater control which has many different types of risk and errors.
Critical weaknesses
Please see below for enumerated points. Major topic 1 concerns novelty against typical routing works and how this method improves upon it. For example, it seems conceptually similar to routing networks, if so, is the novelty in the underwater robot testing? Major topic 2 concerns evaluation and potential limitation of the approach. It is hard to understand for now, how this approach can handle even slight distribution shift in the task, as the task is trained and tested on the same tasks. In addition, this approach seems to be only possible with training in simulation, how would this scale to real world?
- To improve
- (major topic 1) Evaluation / Explanation of Novelty: How is this different from other router learning algorithms (Routing Networks, 2017) that uses the final task performance? It seems conceptually the same as what is proposed, but in the proposed work, it is tested on underwater robots. Clarifying this difference and adding a baseline to showcase the delta will be important. This is consequential as for now, it is unclear the novelty of the paper, both qualitatively and quantitatively.
- (major topic 1) Evaluation: Adding a simple rule and threshold-based baseline (as this is from the introduction), one main paradigm this method would like to move away from.
- For example, when risk is above a threshold (0.7), trigger all three modules.
- (major topic 2) Evaluation: The current evaluation seems very in-distribution. As the method learns with data from the same task (while they are “different states”). I would like to see the split be based on having different training and testing task within a task family, such that the evaluation is not tested with the same task. Please also describe how the randomization between tasks is configured.
- (major topic 2) Evaluation/Approach Question: How would this operate in the real world, where it is not possible to generate such matched-state counterfactual supervision? At minimum, a paragraph of explanation is needed, especially how far the difference between sim and real can be. It would be very helpful if there are real-world deployments of the method with simulator-trained router.
- Motivation: It is hard to understand what makes the approach unique for underwater robots. Some explanation of this will be helpful.
- Explaining motivation: Why is the FP32 detail important? It is repeated throughout the paper. What is unique about the approach that makes preserving FP32 now achievable that was previously not.
- The writing was quite hard to parse, especially on the approach and experimental section. For example, the Figure 1 caption was hard to understand as it had many abbreviations. I would highly recommend structuring the paper such that the novelty is highlighted, and in the approach, describe the approach intuitively. Having some running examples would also help.
Questions for authors
- How is this different from other router learning algorithms (Routing Networks, 2017) that uses the final task performance? Clarifying this difference and adding a baseline to showcase the delta will be important.
- How does this work compare quantaitavley to a simple rule and threshold-based baseline? For example, when risk is above a threshold (0.7), trigger all three modules
- How does this work perform when test set is less in-distribution? I would like to see the split be based on having different training and testing task within a task family, such that the evaluation is not tested with the same task. Please also describe how the randomization between tasks is configured.
- How would this operate in the real world, where it is not possible to generate such matched-state counterfactual supervision? At minimum, a paragraph of explanation is needed, especially how far the difference between sim and real can be. It would be very helpful if there are real-world deployments of the method with simulator-trained router.