- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies off-policy evaluation in finite MDPs where reward recording depends on the unseen reward, so policy values are only partially identified. Under a bounded selection-odds model with parameter , sharp cellwise bounds are respondent expectiles. Assuming all policies share one rectangular reward law, a policy contrast’s sharp width equals the sum of occupancy contrast times reward ambiguity, and auditing a cell removes exactly its term regardless of the revealed outcome. Audit selection therefore reduces to top- or knapsack. A split-sample sign-envelope procedure gives outer confidence bands robust to near-zero occupancy contrasts, evaluated in a controlled MDP, semi-synthetic KuaiRec, and two field follow-up studies.
Critical strengths
Some strengths of the paper are as follows:
-
The identification results are correct as far as I could check them. I verified the two-cell example, the binary expectile endpoints, and the selective-success counterexample showing that intersecting projections are not sharp. This augments the paper’s most reusable contribution that missing rewards can change a policy decision.
-
The shared-law formulation makes clear why separately bounding each policy value can be unnecessarily conservative. The reward uncertainty in cells visited by both policies may cancel in their difference. The paper connects this observation to sequential occupancy measures, where the relevant weighting depends on discounted state–action visitation rather than simply on local action disagreement.
-
The paper reports whole-set coverage rather than relying exclusively on pointwise coverage and includes Monte Carlo uncertainty in its simulation results. It also distinguishes identification-width improvements from sampling precision and discusses the limitations of the field-study analyses.
Critical weaknesses
The weaknesses are stated below:
-
The theoretical contributions seem incremental. The principal sharp-width identity is fine, but under Assumption 2 it follows naturally from optimizing a linear functional over a rectangular set of admissible reward means. The additive audit gains and the top-(K) or knapsack allocation rules then follow directly from that identity.
-
Proposition 3 applies a known expectile duality for odds-ratio sensitivity models. The authors themselves acknowledge that cancellation, and its advantage over separate subtraction, were already derived by Guerdan et al. (2024). What is the novelty then in this one?
-
The paper’s main claims are not tested on real sequential data, and the field studies mainly show that the key assumptions are somewhat inconsistent. The YOP analysis shows that the audit-validity assumption (recovering the original outcome at the original time) fails. The Coppock recovery depends on outcome stability across a fourfold incentive change.
-
KuaiRec is sort of semi-synthetic. The response mask is imposed, and the policies are constructed. So no real setting demonstrates nonignorable reward recording, recoverable audits, and a sequential policy comparison together.
Questions for authors
Here are the questions for the authors. If they are addressed, I am willing to raise my score.
- Given Assumption 2, Theorem 4 follows from optimizing a linear functional over a box; then what is new beyond Guerdan et al. (2024)?
- It seems that, in the controlled MDP, the reference policy is the behavior policy Unif? The cancellation ratio ranges from 11.5 × to 51.5 × across behavior mixtures, so how much of the headline is overlap by construction?
- Is the sign-envelope needed? In Tables 13–14, the ordinary centered-endpoint sup- band attains 94.0–100% coverage, including at exact-zero occupancy pairs. Can you exhibit a DGP where ordinary endpoint inference materially undercovers and the sign-envelope corrects it?
- Is there any dataset with genuine reward-dependent recording, a sequential policy comparison, and recoverable outcomes (for example, delayed-conversion logs)?