- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper looks at the problem of language-conditioned RL. It introduces GLEAN, a method of incorporating trained subsequences with sub-instructions into a method by training a policy to follow each subgoal, a detector to see when subgoals are completed and online RL. The paper shows results on BabyAI against several baselines.
Critical strengths
Despite there being a lot of work in language-conditioned RL, the method does seem to be a new contribution and the related work section mostly does a good job of positioning the work (although as a minor nit, I think some older works, especially Policy Sketches may have been missed).
The method is evaluated on a variety of strong baselines which are introduced and explained well in the related work including HRLLI, CAREL and HIGhER and Ella, covering a pretty wide variety of language conditioned RL methods.
The methodology seems sound. While intro might have described it better, S4 and figure 1 make the method fairly clear and explain how the method works clearly.
Critical weaknesses
The biggest weakness to me are the experimental validations:
- First, the method is only validated on BabyAI which is a bit old and a bit toy. This is the only environment the method is run or compared on, so it's a bit difficult to tell how well this method really generalizes. For instance, would the completion detector work nearly as well on environments that are not just 2D grids (e.g. 3D environments, video or even real world settings). This is a big part of the method and I am not sure BabyAI is the best test for it.
- While the number and recency of baselines is impressive, the results themselves make the usefulness of the method slightly mixed. It seems that HIGhER+BC is more effective on half of these environments. The statement about where the paper's method is better: "achieves higher final success than every baseline that receives a completion signal as reward" is not clear to me. It would be useful for authors to explain further what they mean and say where they think their method has advantages over others, especially HIGhER+BC.
The data-budget experiment is appreciated, but if it's important it really ought to be in the main text of the paper.
Questions for authors
See above: Can you explain further what you mean and say where they think their method has advantages over others, especially HIGhER+BC? I am also having trouble understanding what you mean in your abstract when you describe where you method is better. Can you explain this?