acceptodds
Under review as a conference paper at ICLR 2027

Learning the Observation Function in POMDPs: Identifiability through State Invariance

Abstract

Policies for partially observable Markov decision processes (POMDPs) act on state observations. The observation function governs the probabilities of emitting observations at each state. We study the following problem: Identify the unique probabilities of an unknown observation function from a dataset of observations generated under an unknown policy. However, the underlying observation function of a POMDP is not identifiable in general. We therefore define a class of observation functions with the key property that observation probabilities are identical across all states that can emit them, for example where the probabilities of sensor failures are invariant, that is, consistent across states. We call this class of observation functions state-invariant and show them to be identifiable. We provide an optimization-based approach to learn the observation function from data: we start with a natural formulation of the problem as a nonlinear program (NLP) and convexify the NLP toward a disciplined convex program (DCP) solvable in polynomial time. We prove that the solutions to the optimization problems are probably approximately correct (PAC) with respect to the dataset. Our experiments support the theoretical findings.

Primary area:
reinforcement learning
Keywords:
POMDP, MDP, PAC, Missing data, Identifiability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SoVXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies identification of an unknown POMDP observation function from observation-only trajectories generated by an unknown behavior policy. Under a known state–observation compatibility relation and a state-invariance assumption, it proposes an identification method, a convex relaxation, and computational improvements.

Critical strengths

The main strength is the observation that shared emission probabilities and normalization constraints can identify sensor parameters despite unknown state visitation. This is a meaningful objective distinct from learning a near-optimal policy or a predictive representation. The experimental results suggest that the proposed formulations can be practical.

Critical weaknesses

The theoretical proof require some changes. In Appendix C.2, the derivation of Equation (22) incorrectly assumed \hat{q}_z is independent of z. Section 5.1 and Appendix D establish convexity of the relaxation but do not prove that it preserves identification and PAC guarantees. These issues appear potentially fixable, but the required proof revisions and their verification is beyond the scope of intended author rebuttal. I therefore recommend rejection and resubmission with a complete, revised analysis that can be properly reevaluated.

Questions for authors

Should Theorem 4.6 claim uniqueness only of the emission probabilities? How do the proofs handle observations that have positive probability at different time steps? Should the bounds in Equations (17)–(19) include a dependence on the horizon L? Please check the sign in Equation (14) and the confidence probabilities in the proof of Theorem 4.8.

2 comments

We thank the reviewer for the exceptionally careful reading of our proofs. The questions identified several mistakes in the proofs, all of which are now fixed. The fixes only have “local” impact, meaning that the stated results continue to hold.

W1: Equation (22) incorrectly treats q^z\hat q_z as independent of zz.

The reviewer is right. Please see (G1).

W2: The convex relaxation lacks identification and PAC guarantees.

The revised paper now explicitly provides both. Please refer to (G2).

Q1: Should Theorem 4.6 claim uniqueness only of the emission probabilities?

The reviewer is right. As noted in lines 280--281 (in the original PDF), we have no guarantee on the individual pp variables. The revised statement says that (p~,q~)(\tilde p,\tilde q) is a global optimum and that every global optimum (p^,q^)(\hat p,\hat q) satisfies q^=q~\hat q=\tilde q.

Q2: Observations with positive probability at different time steps.

Thank you for spotting this oversight in the proof. In Lemma 4.5, identity (13) holds for each fixed tt, but the time step tt with cz(t)>0c_z^{(t)}>0 may differ across observations zz. We close the gap by adding over time inside the proof, similarly to the time flattening we already do in Section 5.2. Since (13) holds for every tt, it also holds for the time-summed quantities. Please refer to the updated PDF for details.

Q3: Should equations (17)--(19) depend on the horizon LL?

Yes. Objective (2) sums over all tt, so the correct bound is ∑t∑z(ez(t))2≤L∣Z∣(ε′)2\sum_{t}\sum_z (e_z^{(t)})^2\le L|Z|(\varepsilon')^2. In the revised proof, this factor LL appears explicitly. Because the proof now works with time-flattened quantities (see Q2), the factor is matched by a factor LL on the other side and cancels out.

Q4: Sign in equation (14) and confidence levels in the proof of Theorem 4.8.

Thank you; both are corrected. The sign in (14) was flipped, but the proof is unaffected, as (15) equates it to 0. In the proof of Theorem 4.8 (and Lemma D.1), "with probability at least δ\delta" should read "at least 1−δ1-\delta".

On the proof revisions being beyond the scope of the rebuttal.

We fully understand the reviewer's concern that proof revisions need careful checking. We would like to emphasize that these mistakes (and their fixes) are local: the claims in the paper remain the same, and we have fixed all points. The fixes are brief and uncomplicated. We have highlighted every change in the revised PDF to make reviewing it as efficient as possible, and we would be grateful if the reviewer would consider the revision.

Thanks Reviewer SoVX again for the feedback. We hope our rebuttal has adequately addressed your concerns, and we would be grateful if you would reconsider your score. If any concerns remain, we would appreciate if you could let us know, and we would be very happy to provide further clarification.

Reviewer_MEiCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper aims to learn the observation function p(z|s) in a POMDP setting, where z is an observation and s is a state.

Learning it from observation data alone is intractable because there are many possible p(z|s) that are consistent with collected data from an unknown behavior policy \pi and unknown transition dynamics p(s'|s, a).

E.g., at a particular time t, fixing p(s), in reality, this is also unknown, p(z1) = p(z1|s1)p(s1) + p(z1|s2)p(s2); plugin values 0.2 = 0.6·p(z1|s1) + 0.4·p(z1|s2),so there are infinite combination of p(z1|s1) and p(z1|s2).

The idea is to assume state invariance; under this assumption, p(z1|s1) = p(z1|s2) = q_z1, i.e., the emission probabilities for z are the same across all states.

Under this assumption, the problem becomes tractable (efficiently identifiable). The authors then formulate finding q_z as an optimization problem with finite samples and show that the solution converges to the true solution with N->inf.

Then, the authors propose a convex formulation for the optimization problem to get a solution in polynomial time.

Critical strengths
  1. Generalizing missing at random (MAR) to state invariant (SI) in POMDP and showing that with this assumption the observation function becomes identifiable. Formulating the identification of observation function as non-linear program and showing it's convergence to true solution and PAC.

  2. Convex relaxation of the non-linear program to obtain solutions in poly time and other methods like iterative solving and pruning to further decrease run time.

Critical weaknesses
  1. The assumption of state invariant is quite strong. It covers, particular setting where the sensor observation either gives exact reading or fails and the failure is dependent on the observation (not on the state).

  2. It is mentioned that NLP is relaxed in disciplined convex program which inherits the PAC guarantees. However, this is not shown in the paper. When number of samples tends to infinity, the solution of DCP will converge to true solutions, but for PAC, the rate also matters and it could be different for NLP_main and DCP_main. Indeed, DCP will inherit PAC, however, with different rates, which needs to be shown and/or discussed. 

  3. The PAC bounds should be discussed explicitly and describe it as an instance-dependent guarantee since it depends on the least-visited states under the unknown behaviour policy and can be exponentially small, which would make the PAC bounds very loose and uninformative.

  4. Some missing/unexplained evaluation:

  • In Fig. 2a,b, the results of NLP-main are omitted, and in Fig. 2b, the results of NLP-iterative are omitted. Since there are only two environment benchmarks, it would be better to show evaluation (MSE) results for all techniques in Figs 2 a,b. It is understandable to skip, time evaluation of some as mentioned in the paper, but better to put MSE for all.
  • In Fig. 2a, DCP_flat performs better than NLP_flat as the dataset size increases. This is not discussed and is quite unclear, since theoretically NLP_flat should perform better than or equal to DCP_flat?
  • The error gap between NLP_iterative and NLP_flat seems quite large; what about DCP_iterative and DCP_pruned? Section 5.3 focuses on justifying iterative solving and pruning and mentions: "Both variants admit convex relaxations analogous to Section 5.1," but the experiments do not clearly establish this comparison in terms of the MSE loss in objective. A faster runtime is understandable, but the trade-off with the error in the objective needs to be established.
Questions for authors

Q1. Discuss on the missing and unexplained evaluation in Figs 2 a and b (see above).

Q2. The iterative and pruning approach discussed in Section 5.3 needs to properly discussed for the trade-off between run time and MSE. It seems, not performing well, so Section 5.3 approach is futile and maybe suitable for appendix (as ablation).

Q3. Will Bernstein's inequality tighten the sample complexity in PAC?

4 comments

We thank the reviewer for the clear summary and the constructive feedback, especially on the experimental evaluation.

W1: Strength of the SI assumption.

Please see (G4), where we explain that SI is not limited to sensors that either give an exact reading or fail.

W2: PAC guarantees of the DCP and their rate.

This is an excellent point. The revised paper now gives an explicit PAC bound for the DCP. The reviewer's intuition is correct: the relaxation inherits PAC identification but with a worst-case rate that is worse, ε−4\varepsilon^{-4} compared with ε−2\varepsilon^{-2} for the NLP. Please refer to (G2) for details.

W3: Instance dependence of the PAC bounds.

We agree and now describe the guarantees explicitly as dependent on the instance (POMDP parameters and behavior policy), please see (G2).

W4 / Q1: Missing evaluations in Fig. 2a/b.

The reviewer is right that the current plots can give the impression that results are being omitted. We had treated NLPflat\mathtt{NLP_{flat}} and DCPflat\mathtt{DCP_{flat}} as our "final products", and viewed the others as intermediate steps. We agree that a complete picture is more informative.

We are currently running NLPmain\mathtt{NLP_{main}} for Fig. 2a/b, NLPiterative\mathtt{NLP_{iterative}} for Fig. 2b. We will upload the updated figures before the end of the discussion period, and post a short comment when they are available. We are posting the global response before these results are in to ensure enough time for discussion. We do not expect the intermediate variants to outperform the final ones, but showing this explicitly completes the comparison.

W4 / Q2: DCPflat\mathtt{DCP_{flat}} vs. NLPflat\mathtt{NLP_{flat}} in Fig. 2a.

Thank you for raising this. The guarantee that the NLP is at least as accurate applies to global optima in the limit. For finite data, the two programs minimize different objectives, and neither dominates the other on every instance (our worst-case bound for the DCP is in fact weaker). Figure 2a shows the expected behavior: NLPflat\mathtt{NLP_{flat}} has lower error for small datasets, and DCPflat\mathtt{DCP_{flat}} catches up as data grows. At large dataset sizes, both reach MSE in the 10−410^{-4} to 10−510^{-5} range, where the remaining gap depends mostly on numerical error. Specifically, it depends on solver tolerances and stopping criteria (Table 1) and on the solvers themselves. We solve the NLP with a nonconvex solver (GEKKO), whereas we solve the DCP with a convex solver (MOSEK). We added this discussion to Section 6.

W4 / Q3: Runtime vs. MSE trade-off for pruning and iterative solving.

We agree that this trade-off should be stated in more detail. Section 6 (lines 428–430) and Figure 4c (in the original PDF) mention it only briefly. By design, iterative solving trades accuracy for speed. Each program P(t)P^{(t)} fixes the emission probabilities estimated by earlier programs, so their errors compound over time. The higher MSE is therefore expected, while the speedup matches the theory (Fig. 2f). Iterative solving is thus useful when NLPmain\mathtt{NLP_{main}} is too large to solve at all.

W4 / Q4: MSE comparison between DCPiterative\mathtt{DCP_{iterative}} and DCPflat\mathtt{DCP_{flat}}

Parallel with W4/Q1, we are currently calculating the MSE of DCPiterative\mathtt{DCP_{iterative}}, and we will compare it against DCPflat\mathtt{DCP_{flat}}. We will update the PDF once we obtain the results. The iterative problem formulations already include the pruning in NLPpruned\mathtt{NLP_{pruned}} and DCPpruned\mathtt{DCP_{pruned}} and therefore do not evaluate these two formulations in our benchmarks.

Q5: Would Bernstein's inequality tighten the sample complexity?

We believe it would. Our aim is to define a PAC-identification problem (Section 4.1) as general as possible and show that SI observation functions are PAC-identifiable under minimal assumptions. We therefore opted for the simplest bound that suffices. Our use of Okamoto's inequality is essentially Hoeffding's inequality for Bernoulli variables. A bound that takes variance into account, such as Bernstein's, would mainly improve the dependence on rarely observed observations, whose empirical frequencies have small variance. More generally, stronger inequalities can be used when more is known about the model, yielding tighter bounds.

Thanks for your answers.

While i dont absolutly oppose accepting the paper, I am not particularly excited by the results.

I thus keep my weak reject score.

Thank you for the reply. Would you kindly let us know which points of your review need further clarification? We have already tried to address the strength of the SI assumption, the sample complexities of the PAC guarantees and the details of the experimental evaluation, but we are of course happy to explain these issues further or receive more feedback on how to improve our work.

We have uploaded a revised version of the PDF document with the requested experiments (W4). Please refer to Figure 4e and the comments under Experiment 2 in Appendix H.

Reviewer_zDwdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the identifiability of unknown observation functions in partially observable Markov decision processes (POMDPs) using observation trajectories collected under an unknown policy. The authors introduce the state-invariance (SI) assumption, under which the probability of producing a particular observation is identical across all states capable of generating that observation.

Based on this assumption, the paper formulates observation-function estimation as a nonlinear optimization problem and proposes a convex relaxation for improved computational efficiency. The authors further introduce time-flattened and sparsity-aware variants, together with theoretical PAC-identification guarantees. Experiments on synthetic POMDPs and a drone navigation example demonstrate improvements in estimation accuracy, optimization efficiency, and downstream planning performance.

Critical strengths
  1. An interesting theoretical formulation of observation-function identifiability. Learning observation models from partially observable trajectories without knowing the behavior policy or transition dynamics is an important problem. The proposed state-invariance assumption provides a potentially useful connection between POMDP identification and the missing-data literature. Establishing identifiable model classes is valuable for understanding when model-based learning can recover the underlying environment rather than merely fit observed trajectories.

  2. An attempt to connect theoretical identifiability with computational tractability and planning. The paper goes beyond a purely theoretical statement by developing optimization formulations, investigating computational improvements, and examining the effect of observation-model estimation on downstream policy performance. The combination of identifiability, efficient optimization, and decision-making consequences is a meaningful direction.

Critical weaknesses
  1. There appear to be gaps in the proofs supporting the main theoretical guarantees. In Appendix C.2, the derivation from Eq. (21) to Eq. (22) appears invalid: an equality involving sums over observations is multiplied by a quantity that depends on the summation index, (\hat q_z), without justification. This step is subsequently used to establish the approximation bound in Lemma 4.7, which supports Theorem 4.8. In addition, the claimed PAC guarantee for the convex relaxation needs a more explicit justification, including how the observation probabilities are recovered from the relaxed solution. Since theoretical identifiability and efficient PAC estimation are the central contributions, these issues materially affect my confidence in the paper.

  2. The assumptions and experimental settings substantially limit practical applicability. The method requires the state-observation admissibility relation to be known and the observation function to satisfy exact state invariance. These assumptions are restrictive for many realistic partially observable systems, particularly those involving continuous states, complex visual observations, or state-dependent sensor noise. The experimental evaluation is primarily based on synthetic finite-state systems and a constructed navigation problem, leaving the applicability of the results to more general POMDP learning settings unclear.

Questions for authors
  1. Can the authors clarify the derivation from Eq. (21) to Eq. (22) in Appendix C.2? Since (\hat q_z) depends on (z), multiplying the summed equality by this quantity does not appear algebraically valid. Can a corrected proof of Lemma 4.7 and the resulting PAC guarantee be provided?

  2. Can the authors give a formal justification for the PAC guarantee of the convex relaxation, including a precise procedure for reconstructing (q_z) from the optimized state distributions? How do the guarantees depend on small observation probabilities and the conditioning of the admissibility structure?

  3. How sensitive is the proposed method to violations of the known admissibility relation or the exact state-invariance assumption? The current noise experiment is useful, but understanding whether modest structural misspecification causes systematic estimation errors would substantially affect the practical significance of the approach.

2 comments

We thank the reviewer for the thorough review and for the precise pointer to equations (21)--(22).

W1 / Q1: Gap in the proof of Lemma 4.7 (Eqs. 21--22).

The reviewer is right. Please see (G1) for the corrected argument. The full proof is in the revised Appendix C.2.

W1 / Q2: PAC guarantee for the convex relaxation, recovery of qzq_z, and dependence on small probabilities and the admissibility structure.

Please see (G2). The emission probabilities are recovered as q^z=cz/∑s: z⪯sp^s\hat q_z = c_z/\sum_{s:\,z\preceq s}\hat p_s. The revised paper gives explicit sample complexities for both the NLP and the DCP. As the reviewer suspected, the bounds depend on the smallest emission probability qmin⁡q_{\min} and on the smallest probability of visiting a state that can emit a given observation Pˉmin⁡\bar P_{\min}. We now explicitly discuss this bound and its dependence on the instance. This dependence is unsurprising because we aim to provide the most general statement possible. In applications with more knowledge about the model and behavior policy, for instance, where a lower bound on the state visitation probabilities is available, it would be possible to provide tighter bounds.

W2: Restrictiveness of the assumptions.

Please see (G4). In short, assumptions are necessary for identification (Appendix A). SI strictly generalizes MAR, the standard assumption in the missing-data literature, and it also covers error signals and state-dependent sensor behavior. Regarding the experiments: measuring identification error requires the ground-truth observation function, which is unavailable for real-world data. Synthetic benchmarks with a known OO are therefore the natural way to evaluate identification. The drone case study complements them by showing the effect on planning.

Q3: Sensitivity to violations of ⪯\preceq and of exact SI.

Thank you for this suggestion. We added the requested experiment on misspecified admissibility relations (see G3). Together with Figure 2g on violations of exact SI, it shows that the error grows gradually with the degree of misspecification rather than causing a sudden complete failure.

Thanks Reviewer zDwd again for the feedback. We hope our rebuttal has adequately addressed your concerns, and we would be grateful if you would reconsider your score. If any concerns remain, we would appreciate if you could let us know, and we would be very happy to provide further clarification.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response (1/2)

We sincerely thank all three reviewers for their constructive feedback. Several comments were exceptionally detailed and helped us substantially improve the paper. We have uploaded a revised PDF with changes highlighted in green.

We are glad the reviewers found the problem itself well motivated.

  • zDwd and SoVX both stress that identifying the observation function is an important objective in its own right.
  • zDwd and MEiC appreciate that state invariance (SI) generalizes the class MAR (missing at random) from the theory of missing data and connects it to POMDP identification.
  • zDwd and SoVX note that the experiments indicate the approach is practical.
  • All reviewers (zDwd, SoVX, MEiC) value that the paper goes beyond an identifiability statement by also providing PAC correctness and exploring computational optimizations.

Summary

We have made the following changes:

  • We have addressed all technical points from the reviewers in the uploaded revision. We emphasize that the fixes were local, brief, and uncomplicated, and that all results remain unchanged.
  • We have added an experiment where the relation ⪯\preceq doesn't structurally match the ground truth, showing that error gradually increases with the proportion of mismatches in the relation.
  • We have included PAC bounds for the DCP, with explicit sample complexity bounds for both the NLP and DCP in the main text, showing that the computational speedup of the DCP comes at the cost of a weaker worst-case sample complexity.

Below, we address the common concerns. Additionally, we respond to each reviewer’s specific questions individually below their respective comments.

(G1) [zDwd, SoVX] Oversight in equations (21)--(22) in Appendix C.2.

Both reviewers are right, and we thank them for spotting this oversight in the appendix. Going from (21) to (22), we multiplied a sum by q^z\hat q_z, which depends on the summation index. This was a slip in an intermediate step. Lemma 4.7, and therefore Theorem 4.8, still holds. The revised proof avoids the step entirely, and we refer to Appendix C.2 in the updated PDF for the full derivation.

With Pz:=∑s:z⪯spsP_z := \sum_{s: z\preceq s} p_s and P^z:=∑s:z⪯sp^s\hat P_z := \sum_{s: z\preceq s}\hat p_s, we expand Pz−P^zP_z-\hat P_z using the true qzq_z instead of q^z\hat q_z, giving us the identity

∑z(qz−q^z)2P^zqz=∑z(qz−q^z)Dzqz,Dz:=qzPz−q^zP^z.\sum_{z} \frac{(q_z-\hat q_z)^2\hat P_z}{q_z} = \sum_z \frac{(q_z-\hat q_z)D_z}{q_z},\qquad D_z := q_zP_z-\hat q_z\hat P_z .

The sampling error and the (small) objective value of the minimizer control DzD_z. We then bound each side directly by lower-bounding P^z\hat P_z on the left and applying Cauchy-Schwarz on the right.

(G2) [zDwd, MEiC, SoVX] Explicit PAC bounds for the NLP and the DCP, now in the main text.

We agree that the convex relaxation deserves an explicit derivation rather than a remark, and that the NLP bound should not be hidden in the appendix. The revised Sections 4.4 and 5.1 now state explicit PAC bounds for both.

Let qmin⁡:=min⁡zO(z∣sˉ)q_{\min}:=\min_z O(z\mid\bar s) be the smallest nonzero emission probability, and let Pˉmin⁡:=min⁡z1L∑t=1LP(z⪯s(t))\bar P_{\min}:=\min_z \frac1L\sum_{t=1}^{L}\mathbb P(z\preceq \mathbf s^{(t)}) be the smallest time-averaged probability of being in a state that can emit zz.

  • NLP (revised Thm. 4.8). If N≥2∣Z∣(1+∣Z∣)2ln⁡(2∣Z∣L/δ)ε2qmin⁡2Pˉmin⁡2,N\ge\frac{2|Z|(1+\sqrt{|Z|})^2\ln(2|Z|L/\delta)}{\varepsilon^2q_{\min}^2\bar P_{\min}^2}, then with probability at least 1−δ1-\delta, every global minimizer of NLPmain\mathtt{NLP_{main}} satisfies max⁡z∣q^z−O(z∣sˉ)∣≤ε\max_z|\hat q_z-O(z\mid\bar s)|\le\varepsilon.

  • DCP (new Thm. 5.2). The emission probabilities are recovered from the DCP solution p^\hat p as q^z:=cz/∑s:z⪯sp^s\hat q_z := c_z\big/\sum_{s:z\preceq s}\hat p_s (cf. line 324). If N≥2048∣S∣∣Z∣2ln⁡(2∣Z∣/δ)ε4qmin⁡2Pˉmin⁡4,N\ge\frac{2048|S||Z|^2\ln(2|Z|/\delta)}{\varepsilon^4q_{\min}^2\bar P_{\min}^4}, then with probability at least 1−δ1-\delta, the solution of DCPflat\mathtt{DCP_{flat}} satisfies max⁡z∣q^z−O(z∣sˉ)∣≤ε\max_z|\hat q_z-O(z\mid\bar s)|\le\varepsilon.

We would like to make two remarks:

  1. In line with what MEiC suggested, the DCP formulation has a worst-case rate (ε−4\varepsilon^{-4} instead of ε−2\varepsilon^{-2}) for the NLP. This is consistent with our empirical observation that the DCP needs more data before catching up with the NLP (Figure 2a).
  2. As zDwd and MEiC point out, both bounds are dependent on the instance (POMDP parameters and behavior policy), in that they scale with how often the behavior policy visits states that can emit a given observation. We now state this property explicitly. Knowing more about state-visiting frequencies may yield tighter bounds. Our aim is to provide weak assumptions that allow identification, to capture a broad class of POMDPs. Moreover, experiments show that practical convergence (Figure 2a/b) is considerably faster than the theoretical worst-case.
AuthorsOpenReview ↗

Global Response (2/2)

(G3) [zDwd] Sensitivity to a misspecified admissibility relation.

Following zDwd's suggestion, we added an experiment in Figure 4d in Appendix H in the updated PDF. We change the relation ⪯\preceq between states and observations, which is given to the algorithm by flipping a fraction κ\kappa of its entries. Flipping ⪯\preceq from 0 to 1 adds unnecessary support, which is manageable in the optimization problems, while flipping it from 1 to 0 removes relations that actually exist and can be more detrimental. We then measure the MSE of DCPflat\mathtt{DCP_{flat}} against the ground truth. The error stays low for small κ\kappa and grows gradually as κ\kappa increases. Together with the existing experiment on violating exact SI (Fig. 2g), this shows that the method degrades gradually under modest misspecification of both kinds.

(G4) [zDwd, MEiC] Strength of the SI assumption.

We stress that identifying the observation function in POMDPs is generally impossible: assumptions are necessary for identifiability.

Among identifying assumptions, missing at random (MAR) is arguably the most widely used in the missing-data literature (Rubin, 1976; Schafer & Graham, 2002; Little & Rubin, 2019), and it underpins standard likelihood-based and imputation-based methods in that field. SI strictly generalizes MAR (Lemma B.2), so every MAR setting is covered.

SI also goes beyond missingness and beyond "exact sensor reading or failure":

  1. Partial readings. In Figure 1, the drone may observe only which quadrant of the grid it is in, with no information about occlusion. This is SI.
  2. Varying levels of detail. Depending on occlusion, the drone might observe its quadrant, exact location, or an intermediate level of detail. The occlusion level itself need not be observed for this to be SI.
  3. Abstract labels. In Example 2, the observations z1,…,z4z_1, \dots, z_4​ are plain labels with no additional structure, rather than sensor readings of the underlying state, yet the model is SI. The example could be extended to a POMDP planning problem where observations are plain labels that serve as clues for achieving some goal.

We agree that alternative settings brought up by the reviewers, such as continuous state spaces, rich visual observations, and approximate SI, e.g., ∣O(z∣s)−O(z∣s′)∣|O(z\mid s)-O(z\mid s')| bounded by some non-zero value rather than zero (see Section 7), are exciting directions for future work. These are currently beyond the scope of the theory.

(G5) [SoVX] Further corrections.

SoVX identified several additional technical oversights, all of which are fixed in the revision. First, Theorem 4.6 now claims uniqueness only for the emission probabilities qq, as intended (lines 280–281). Second, Lemmas 4.5 and 4.7 now handle observations with positive probability at different time steps by aggregating the identities over time. Third, we now make the factor LL in Eqs. (17)--(19) explicit. Finally, we corrected a sign in Eq. (14) and the confidence levels (1−δ1-\delta) in the proofs of Theorem 4.8 and Lemma D.1. None of these changes affects the conclusions of the stated results. Details are in our response to SoVX.

We hope our response fully addresses the reviewers' concerns, and we would be happy to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.