- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper discusses the limitations of rectangualrity assumption of RMDPs where the assumption is overly conservative. They provide theoretical results explaining the gap and also propose a selection algorithm to globally balance which dependencies to restore accounting for interactions between groups that simpler rankings might miss.
Critical strengths
- The paper addresses a genuine weakness of rectangular uncertainty sets: they can combine individually plausible transitions into an impossible joint model. Quantifying the resulting pessimism is useful for robust control and safety-critical decision-making.
- Theorem 1 decomposes the release gap into nonnegative local defects weighted by coherent-model state–action occupancies. This provides a precise explanation of where rectangularization pessimism comes from, rather than relying solely on empirical comparisons.
- The certificate pools positive and negative endpoint preferences across selected blocks. This is important because a block may have zero value in isolation but contribute substantially when selected alongside another block. The paper provides a simple example where the individual certificates are zero, but the joint certificate is positive.
Critical weaknesses
- How reliable is the method when the true factor structure is unknown or only partially specified? Can the authors detect misspecification or provide guarantees under uncertainty about the sharing map itself?
- How much of the proposed method extends to continuous factor spaces, nonlinear parameterizations, infinite-horizon problems, or policies that adapt to information about persistent latent factors?
- Are there practically meaningful additional assumptions, such as stronger mixing or more refined reachability bounds that give better horizon dependence?
- Can the certification procedure handle general categorical transitions and uncertainty in the loss function without losing its practical usefulness?
Questions for authors
See weaknesses