- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents techniques for solving reguarlized Bellman optimality equations, with the main result being an optimal bound with O(1/\epsilon^2) sample complexity, in the desired approximation error \epsilon, improving on previously known rates that scale with O(1/\epsilon^4). The authors also propose a multilevel wrapper around any prior algorithm to boost its error guarantee from an exponent of p to 1+p/2 in \epsilon.
I am unable to give a more intelligible summary, however, since the paper is an AI generated proof summary of an AI generated proof. There is no exposition, and no attempt to make this work human readable.
Also, this is a highly niche topic in a generative model that is only of theoretical interest since it is not applicable to practical problems in an obvious manner. It is not clear to me if ICLR is even a sensible venue here.
Critical strengths
The main 1/\epsilon^2 guarantee as well as the wrapper around black-box algorithms looks interesting and novel. The coupling techniques used to achieve this is also very nice.
Critical weaknesses
The paper has a bibliography of 7 related works on a well-studied topic.
The work reads like a proof and a summary of that proof. The main text does little to actually develop an exposition that explains the problem, its significance, its connection with the broader field, a buildup of the key techniques and a deconstruction of the key results in helping the reader understand the work. As submitted, this is not what a technical paper is supposed to read like.
There is also the issue that the importance and significance of the question being asked is not so clear, since this is a highly niche topic that does not easily connect to practice due to the use of generative model. Note that the generative MDP model has existed for 20+ years since the foundational work of Kearns, Kakade and others, but rarely translated into practically usable insights.
Questions for authors
See the weaknesses above.