- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a framework for estimating and propagating model-uncertainty in latent-models' rollouts.
Critical strengths
The increasing popularity of so-called "World Models," and tremendous success of model-based RL, make a functional and effective uncertainty estimation in latent dynamics models a significant contribution.
The results seem promising.
Critical weaknesses
Presentation and clarity:
The clarity of the presentation, along with the writing, are poor to the point where it is impossible for me to follow the paper. Specifically:
-
Unless I'm mistaken, a large number of terms are used without any definition. A few examples:
- The symbol \hat is used, but it's never explained what it denotes.
- is used, presumably to denote a value function, but is never defined or introduced.
- is a notation used but never defined. I'm not sure what it relates to.
- "Let z = .. be a sufficient latent state", sufficient for what?
- "In environment e": what an environment (MDP, presumably?) constitutes was never defined.
- Aleatoric and epistemic uncertainty are used in one of the equations, but never introduced or discussed. For a paper that focuses on uncertainty quantification in latent dynamics models, I think that this is a major presentation problem.
-
Many of the phrasings are very unclear, to me. A couple examples:
- "Uncertainty is useful for decision making only when its meaning survives this propagation." I'm not sure what this means, and respectively, if I agree with it. For example, as long as the uncertainty is value uncertainty, I think it has meaning.
- "Marginal calibration on replay data does not establish coverage on adaptively selected imagination trajectories, and a high-probability interval does not by itself control expected return." Again, Im not sure what this means. Surely, coverage of replay data gives us a notion of coverage in latent space?
-
There is no background section establishing terms on which the paper's presentation can later build.
-
The text reads in a way that makes me suspect that much of it was written by AI. Specifically: many sentences are composed of multiple terms that sound grandiose, it's not clear what they mean in context, and do not make a lot of sense together. When they could make sense together, I have to speculate what the writer could have meant, rather than be confident about the meaning communicated. For example: "Conversely, replacing that constant with independent factor-wise constants is generally invalid: errors can cross factors at every transition. A sparse system can also be unstable. The relevant object is therefore a matrix of bounded influences, together with the location and size of residual uncertainty." None of these ideas are introduced properly, and as a result, I have no idea what is meant here, and whether it makes sense or not.
Soundness:
As a result of the presentation issues, I am not able to evaluate soundness. A specific example: The transition distribution over the latent state is modeled as a function of . Unless Im mistaken, is never defined. In RL, presumably means the action value function. Why would the transition function depend on the value function (if it even is the value function)? Not explained or motivated (nor are all the terms defined).
Novelty:
Similarly, due to the presentation issues, I am not able to evaluate novelty. Specifically, there is a large body of work around estimating, propagating, and using uncertainty in model-based / RL. Since the presentation of the contributions is unclear, and the presentation of prior work is very limited, condensed, and I don't find it clear, I am unable to establish clear novelty for this work.
Statistical significance:
No statistical significance metric appears to be used in the results.
Questions for authors
I don't have specific questions, exactly. I think the paper needs to be re-written from the ground up, with a focus on clearer writing, definition of all technical terms and notation used, perhaps a background section, etc.