- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a decision model, where the DM can consult an information assistant (IA) for (epistemically interpreted) information about the states before making their final decision. Concretely, the IA proposes a set F of states interpreted as "the true state of the system is within F" and the DM can pick their final action based on this info. Compared to earlier similar work, the paper proposes two generalizations:
- The DM can potentially also pick non-singleton terminal acts.
- The DM does not have to pick from F, even if the IA provided this information.
The paper argues that other models (not allowing for 1. and 2.) face severe practical limitations and therefore need to be adapted. The paper does so by proposing ways to construct specific selections rules that overcome these limitations. Concretely, it is shown how a reward function for the IA can be constructed that is aligned well-enough with the DM's utility function and a specific criterion such that the limitations models of traditional models can be avoided. The paper ends with a simulation study illustrating that the problem is actually relevant in certain scenarios (under different of random utility functions of the DM).
Critical strengths
- The paper is clearly written and explains the limitations of exisiting apporaches well. The toy examples help to make the paper very accessible to a broad audience. The problem of aligning IA better with an human DM seems to be very relevant.
-The decision model is mathematically rigorous and specifically addresses the limitations identified. The proposed construction of a reward function based on the DM's preferences and a general decision criterion is elegant and the characterizations given in Theorems 8.1-8.4 are as far as I can see correct.
Critical weaknesses
The paper didn't manage to fully convince me of the practical relevance of the concrete problem studied. The only motivation given for this is via toy examples that are SPECIFICALLY CONSTRUCTED to demonstrate the limitations of the existing models. This alone is not a convincing argument or justification of the proposed generalization. While the simulation study did a small steop in this direction I really feel the paper would benefit a lot from a proper application example.
Questions for authors
- Is it possible to add more motivation why the problem is actually relevant or even a (small) application example?
2 comments
- We thank the reviewer for their comments. We tried to address the concerns regarding motivation and problem relevance. We improved our experimental setting and hope to have addressed them. As these were common concerns across the reviewers, we wrote a common response. We would be glad to answer further questions and welcome any additional feedback.
Thanks
I thank the authors for taking the time to answer. I keep my positive score.