- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors use a longitudinal causal graph to construct a minimal state, then introduce MOSE to select useful observations from the available history. Synthetic environments and two Atari games evaluate the effect of this selection on learning
Critical strengths
The paper addresses a useful question: how much information should a learning algorithm retain? The observation that a minimal state can train poorly motivates a careful study of the tradeoff between state size and learning performance.
Critical weaknesses
-
The minimality result conflicts with Definition 3.7. Consider independent X_t, rewards R_t=X_t, and isolated actions; an empty state is Markov and preserves every optimal action, yet Algorithm G.1 retains X_t. This contradicts Theorems 4.1 + 4.2.
-
Causal MOSE receives an additional gradient update, and synthetic returns vary by orders of magnitude. Matched updates and normalized per-instance returns are needed to establish the contribution of state selection.
Questions for authors
-
Suppose X_t is independent across time, R_t = X_t, and actions affect neither states nor rewards. Every action is equally optimal, so an empty state is Markov and preserves optimal actions under Definition 3.7. However, Algorithm G.1 retains X_t. How is this consistent with the claimed minimality?
-
Do MOSE’s gains remain when all methods receive the same total number of gradient updates? Please also normalize returns within each task before averaging, so reward scale does not determine the aggregate result, and compare against a recurrent model given the same observation history.