- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a hyperbolic reward function parameterization for inverse reinforcement learning. The hyperbolic parameterization has a attractive properties of hierarchical representation and packing dense objects into tight space. The learning algorithm is unchanged and uses the maximum entropy IRL algorithm with Riemannian gradient descent. The authors tested the proposed algorithm in object world and a museum environment against linear reward parameterization and deep IRL. The results show hyperbolic IRL is more sample efficient and better captures feature interaction.
Critical strengths
- Using hyperbolic distance to parameterize reward functions for IRL is an interesting idea. This is a useful prior if applicable to the problem at hand.
- The paper is well written and relatively straightforward to understand for reader not familiar with hyperbolic space.
- Results show better sample complexity of the hyperbolic model compared to euclidean or classic linear model and better captures human behavior in the co_attendance analysis.
Critical weaknesses
- This paper angles towards neuroscience applications. I think the paper would benefit much more from evaluating on a wider range of neuroscience tasks and environments and demonstrating its impact. Furthermore, the paper would benefit from a more in-depth discussion of the type of environment and tasks the hyperbolic model is suitable for and unsuitable for. The current paper only describes itself as suitable for "densely distributed reward-relevant objects". Also are there non-neuroscience environments which this model may be good for?
Questions for authors
- For the object world environment, is expected value difference of 10 good or not good?
- The museum environment is interesting but I wonder whether it breaks the Markovian assumption of normal IRL? I expect human behavior to be history dependent.