- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In meta reinforcement learning, an agent should distinguish between a control representation that captures the current state and varies across time steps, and task information that remains shared across time steps within the same task. The paper introduces STAC, which uses a control encoder to map a fixed window of context into a control representation and a task encoder to map the complete rollout history into a task representation. This approach improves learning and generalization.
Critical strengths
- The paper provides detailed experimental results, visualizations, and architectural ablations that support the effectiveness of the double encoder architecture.
- The method outperforms prior methods on the evaluated benchmark while using a compact architecture.
Critical weaknesses
- The encoder's loss function is not sufficiently discussed or evaluated. It includes contrastive losses to encourage task-representation learning, but the contributions of individual components, such as covariance regularization and goal prediction, are not examined through ablation studies.
- The overall novelty is limited. Compared with prior methods, the paper uses a Transformer-based control encoder with a fixed context window to aggregate contextual information, in lack of further improvement on algorithms or concepts.
- STAC uses goal coordinates and task labels to train its task encoder, while the baselines do not consistently receive equivalent supervision. The extra task information may induce unequally comparison.
Questions for authors
Could the authors provide ablations removing individual task loss components, particularly goal prediction and covariance regularization to specify the influence of extra task information and regularization term, while keeping the architecture and training budget fixed?