I do not really get what the paper is trying to do. I should be able to follow the material, but I cannot see the big picture in the thicket of developments. Maybe the paper needs the space of a journal to make its point. I will detail my issues below.
Abstract: “can preserve every action gap on its own, yet produce a strict policy cycle through this mixed use” – what does this mean? What do you mean by preserving the action gap? What is the policy cycle? What mixed use are you talking about?
The statement “without knowing their transition, matrix…“ is not clear either
The rest of the section starting from “a learned likelihood study” to the end of the abstract is equally obscure, although there’s an idea what might be said here.
Line 036: is that figure 1b?
Line 041: how is the bridge evaluated at 0.6632? Even if there’s no discount I compute a probability of 0.64 of crossing it. There’s something substantially missing from the explanation how this is computed.
Explain the success probability \rho. How does one get the formula p^2(3-2p)? Maybe it’s trivial, but it’s not obvious to the reader.
Line 051: “some errors therefore leave every comparison unchanged even when they vary across states“ – what is projected out here? Something is clearly ignored. What is it?
Line 080: “phase lag delays that scaling by one step” - more explanation required.
Lines 083 and 084: the reviewer does not understand the caption for (d)
Line 087: what is a continuation kernel? Define.
Line 094-099, end of section: the reviewer did not understand what the paper talks about here
Line 135, equation (6): what is the link of this statement to subsequent theorems?
Line 146, proposition 4: what does causal solution mean?
Line 173: the word “with“ is missing before the word “actions“
The statement “in the row order“ followed by the tuples is not clear.
Lines 177 and 178: unclear
Line 191 and following: here is something interesting when you talk about evaluation do you mean Bellmann evaluation or something else? It is not clear what you mean by the cycle. Clearly, if there’s a cycle, it means it does not converge. How is it a contraction? Could it be that there are multiple contractions composed with each other. As this seems to be the closest to the title of the paper, it would be very important to resolve this to the reader.
Line 211: and following – I do not really understand equation 13 and what it’s trying to do
Line 314, theorem 8: how do I see from it that the pair’s ordering, including ties preserved?
Line 351, equation 20: what is \eta_g? I don’t think this has been defined.
Line 395, figure 2 caption: it appears that if the purpose of these diagrams had been described earlier, it might be easier for the reader to understand what’s going on.
I appreciate the theoretical contributions of this paper and find its main mathematical results meaningful. The authors' response has also clarified several of my concerns.
However, I share the other reviewers' concerns about the clarity and readability of the presentation. Given the importance of clear communication at a conference, I do not think the current presentation warrants a stronger recommendation.
I therefore maintain my rating.