- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies identification of an unknown POMDP observation function from observation-only trajectories generated by an unknown behavior policy. Under a known state–observation compatibility relation and a state-invariance assumption, it proposes an identification method, a convex relaxation, and computational improvements.
Critical strengths
The main strength is the observation that shared emission probabilities and normalization constraints can identify sensor parameters despite unknown state visitation. This is a meaningful objective distinct from learning a near-optimal policy or a predictive representation. The experimental results suggest that the proposed formulations can be practical.
Critical weaknesses
The theoretical proof require some changes. In Appendix C.2, the derivation of Equation (22) incorrectly assumed \hat{q}_z is independent of z. Section 5.1 and Appendix D establish convexity of the relaxation but do not prove that it preserves identification and PAC guarantees. These issues appear potentially fixable, but the required proof revisions and their verification is beyond the scope of intended author rebuttal. I therefore recommend rejection and resubmission with a complete, revised analysis that can be properly reevaluated.
Questions for authors
Should Theorem 4.6 claim uniqueness only of the emission probabilities? How do the proofs handle observations that have positive probability at different time steps? Should the bounds in Equations (17)–(19) include a dependence on the horizon L? Please check the sign in Equation (14) and the confidence probabilities in the proof of Theorem 4.8.
2 comments
Thanks Reviewer SoVX again for the feedback. We hope our rebuttal has adequately addressed your concerns, and we would be grateful if you would reconsider your score. If any concerns remain, we would appreciate if you could let us know, and we would be very happy to provide further clarification.
We thank the reviewer for the exceptionally careful reading of our proofs. The questions identified several mistakes in the proofs, all of which are now fixed. The fixes only have “local” impact, meaning that the stated results continue to hold.
W1: Equation (22) incorrectly treats q^z as independent of z.
The reviewer is right. Please see (G1).
W2: The convex relaxation lacks identification and PAC guarantees.
The revised paper now explicitly provides both. Please refer to (G2).
Q1: Should Theorem 4.6 claim uniqueness only of the emission probabilities?
The reviewer is right. As noted in lines 280--281 (in the original PDF), we have no guarantee on the individual p variables. The revised statement says that (p~,q~) is a global optimum and that every global optimum (p^,q^) satisfies q^=q~.
Q2: Observations with positive probability at different time steps.
Thank you for spotting this oversight in the proof. In Lemma 4.5, identity (13) holds for each fixed t, but the time step t with cz(t)>0 may differ across observations z. We close the gap by adding over time inside the proof, similarly to the time flattening we already do in Section 5.2. Since (13) holds for every t, it also holds for the time-summed quantities. Please refer to the updated PDF for details.
Q3: Should equations (17)--(19) depend on the horizon L?
Yes. Objective (2) sums over all t, so the correct bound is ∑t∑z(ez(t))2≤L∣Z∣(ε′)2. In the revised proof, this factor L appears explicitly. Because the proof now works with time-flattened quantities (see Q2), the factor is matched by a factor L on the other side and cancels out.
Q4: Sign in equation (14) and confidence levels in the proof of Theorem 4.8.
Thank you; both are corrected. The sign in (14) was flipped, but the proof is unaffected, as (15) equates it to 0. In the proof of Theorem 4.8 (and Lemma D.1), "with probability at least δ" should read "at least 1−δ".
On the proof revisions being beyond the scope of the rebuttal.
We fully understand the reviewer's concern that proof revisions need careful checking. We would like to emphasize that these mistakes (and their fixes) are local: the claims in the paper remain the same, and we have fixed all points. The fixes are brief and uncomplicated. We have highlighted every change in the revised PDF to make reviewing it as efficient as possible, and we would be grateful if the reviewer would consider the revision.