- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies finite-horizon entropy-regularized control and characterizes the relationship between three trajectory laws: , the free Gibbs law, , the law obtained by executing its action conditional on the physical kernel, and , the optimal physical law. However, the paper’s presentation makes it difficult to assess the novelty and implications of its results.
Critical strengths
The authors characterize the decision-relevant content of optimism through a specific auxiliary problem, an exact propagated action criterion, and a finite-tilt variance identity. They derive a full-MDP second/fourth-order separation theorem with explicit coefficients.
Critical weaknesses
The presentation does not clearly distinguish new contributions from established results. When introducing each result in the main text, the authors should identify which parts are established and which are new, outline the main proof ideas, refer readers to the relevant appendix for the full proof, and explain the result’s significance. Currently, the theorems read as a collection of separate statements, with insufficient explanation of how they connect. The distinction between established results and new contributions is discussed mainly at the end of the conclusion section.
The practical implications of the mathematical results also need further explanation and development. For example, when should a practitioner use Gibbs extraction followed by correction, and when should they solve the physical control problem directly? The asymptotic results and confidence bounds provide some theoretical insight, but the paper does not clearly explain how these insights would guide the design or use of a controller in a concrete application.
Questions for authors
-
Second paragraph of introduction, what does "physical value" mean?
-
What are the implications of theorems 5,6,7?