- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors introduce a method called coverage-calibrated Bellman quotients (CBQ) with the aim of better dealing with decision-relevant rare cues, based on a compression of evidence, rather than frequency.
Critical strengths
None
Critical weaknesses
- It’s not clear what advantage this provides over standard predictive modeling of control-theoretic variables. Figure 4 not clear what task this would solve in the real-world, besides be able to predict the red cue, which by definition CBQ would predict. But if the red cue is decision-relevant, then it would be picked up by a neural networks whose weights are optimized in the standard predictive modeling sense. So it's not clear what the practical advantage of CBQ is, and it seems incorrectly portrayed as a limitation of predictive modeling, when in fact it can be covered by it.
It would therefore be important to exhibit a comparison between CBQ and DQN (both with neural network encoders as in Fig. 4) but when the red cue is decision-relevant, and to show that CBQ outperforms (or is more sample-efficient) than DQN.
- The experiments are toy so it would be good to additionally try to demonstrate the advantage of this approach (and exhibit this decision-relevant comparison between CBQ and DQN) to Craftax (or any other visually rich setting).
Questions for authors
-
The writing is not very clear and uses unconventional control-theoretic terms, for example, what is an "atom"? I don't see it defined anywhere.
-
Please address the weaknesses above