- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
In this paper, the authors study how coarsening the information used for inverse-weight normalization affects robust policy certificates under unobserved confounding. The main theoretical result gives an exact expression for the loss caused by grouping histories: the loss equals the increase in weighted pinball risk that comes from forcing histories in the same group to share a threshold. Then, the authors provide a characterization of when grouping is harmless (i.e., the grouped histories share a valid conditional reward quantile). The paper also introduces rectangular augmented certification (RAC), which combines several factors (e.g., simultaneous nuisance envelopes, augmented scores, threshold fitting, independent betting) to obtain finite-sample lower confidence bounds for binary rewards and an always-treat policy. Finally, the experiments are conducted to examine the trade-off between structural loss from grouping and the statistical benefit of reducing threshold complexity.
Critical strengths
- The research question is clear and meaningful.
- The population result is somewhat clean and easy to interpret. The authors also spent time on the finite-sample guarantee, which I appreciate.
- The experiments are with sufficient elaboration and insights. They do more than report final performance.
- The authors keep it transparent where the proposed choices do not help. They chose to report where fixed thresholds can be competitive, and show cases where coarsening helps only slightly.
Critical weaknesses
- My major concern with this paper is that its theory feels somewhat incremental relative to the existing results. The authors did note that the excess-risk identity when interfaces are finite follows by subtracting existing representations from Tan (2024). The paper provides several extensions, but I am not yet convinced that they can lead to something significant enough to justify the weight placed on Theorem 1. I think the paper would be stronger if it showed one concrete consequence that genuinely depends on the new general formulation.
- As far as I can understand, RAC heavily relies on strong assumptions of the nuisance envelopes. The authors did note that learned/misspecified features require further work. This leaves a gap between the motivation and the strongest finite-sample result. The population study with HC-MNIST is useful, but the paper does not demonstrate the full RAC scheme with realistically estimated high-dimensional nuisance functions.
- The benefit of confidence-aware threshold fitting is not fully convincing to me. Clear improvement over empirical-mean fitting is a good result, but I do not think it is enough. A more relevant comparison may be against a strong fixed threshold. Experiments show that fixed zero can be slightly better, and this makes me unsure how important the adaptive fitting stage is to the overall contribution.
- The theory does not yet lead to a practical rule for selecting the normalization interface. Theorem 1 characterizes when grouping loses information, but the experiments mostly evaluate handpicked interfaces. Thus, the paper explains how to measure the cost of a given grouping but leaves open the more practical issue of how to choose the grouping from data (the authors did note this as future work). However, for me, the characterization would have more practical impact if it were used to build a data-driven method to select interfaces.
- The writing prose of the LLM the authors use as writing assistant actually hinders people from reading the paper. Some sections feel like piles of jargon (the Conclusions section for example). The paper would have been much better if the authors had actually put effort into writing it by themselves.
Questions for authors
- How realistic are the simultaneous nuisance envelopes in applications with learned representations?
- Can the shared-quantile characterization be used directly to construct interfaces?
- What should the threshold fitter optimize in practice?
- Can we have the comparison between methods in a setting where their inference assumptions hold?
Thank you for your feedback
We thank the reviewer for the time and care taken in reviewing our paper and discussing the revisions. We appreciate your helpful feedback and positive assessment.