- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work does a systematic characterization about the problems of learning with observation-deletion rewards. Through several examples and environments including GP, Mastermind, and temperature sensor records, the authors analyze the induced policies from such rewards, and importantly, how such polices are suboptimal for the underlying tasks.
Critical strengths
This work is insightful, and the authors did a great job of supporting their arguments with both theory and experiments.
Critical weaknesses
To me, the main takeaway from this paper is that observation-deletion type of reward is not ideal, and the authors did a solid work in demonstrating this. However, it is unclear to me to how broadly such type of reward is used, and additionally, can the related insights extend other forms of rewards.
Questions for authors
How can the findings guide us to a practical reward design? This is a somewhat future-work kind of question, so a broad answer would be fine.