- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies context-based offline meta-reinforcement learning and characterizes the loss when test tasks fall outside the training distribution. The authors further propose VeriCode, which demonstrates empirical success across six Meta-World task families, six MuJoCo task families, and three algorithms. However, the paper’s unclear presentation makes it difficult to assess its technical contributions.
Critical strengths
The paper works on an important problem of test-time task inference under distribution shift in context-based offline meta-reinforcement learning, and it provides a theoretical characterization of performance loss and the proposed VeriCode method.
Critical weaknesses
The presentation and logical flow are difficult to follow. Several terms, assumptions, and methodological details need clearer explanations. For example:
-
The introduction uses ambiguous words and introduces concepts without sufficient context. In the third paragraph, the authors state that “the two benchmarks turn out to fail for different reasons.” Benchmarks themselves do not fail; the authors should identify which algorithms fail on these benchmarks. Similarly, statements such as “On cheetah-vel, extrapolation dominates” and “loss is contextual” need precise explanations. What is being extrapolated, and what does “contextual” mean here? “Regime C” is also mentioned without being introduced. Finally, the statement “This points at procedures that change nothing about training and act only on which code is deployed” is unclear: "This points at procedures" refers to which points?
-
Terminology should be defined when first introduced. For example, “OOD” appears in the introduction without a clear definition. And later in section 3, the authors mention ""OOD" describes the experiments".
-
In section 3, Expressions such as “we make precise the return a deployed context-conditioned policy leaves on the table in an OOD task” are hard to read. is said to be a reachable set, but the definition of a reachable set is missing. Equation (2) appears to follow from a standard finite-horizon performance difference lemma. The authors should cite the relevant literature and clarify which part of the theorem is new. And in Theorem 1, it suddenly mentions policy-sensitivity assumption, action-Lipschitz assumption, but these assumptions are not introduced in the main paper and not referred to the appendix position.
Questions for authors
The paper should provide pseudocode for vericode specifying its inputs, verification steps, code-selection procedure, and output. How does the method apply to general tasks beyond its implementation in Meta-World?