- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors offer an invariance assumption on the copula of the ranks of potential outcomes with and without treatment. Then, they provide a way to optimize an utility function when such an assumption holds using standard techniques from OT and optimization.
Critical strengths
Finds a theoretically sound simplification of an otherwise difficult problem. Full optimal policy learning with no structure on the conditional copula is a difficult objective and in practice would almost certainly result in valid bounds but which are too loose to be useful. The authors propose a simplification that on the theoretical side seems to provide necessary structure for a simple optimization for the policy. Furthermore, this simplification allows for possible sensitivity analysis under violations and the authors provide a method for robust learning.
Critical weaknesses
-
It is unclear how much we have to pay for the robustness. The authors present us with a graph that shows how robust learning performs under correct specification of the violation parameter. However, it is unclear whether assuming a higher level of violation than the correctly specified one would produce a result less satisfactory than the competitors’ benchmark results. Methods of looking at violations are very good while you are in a vicinity of 0, yet deteriorate quickly to the performance of the worst-case policy learner otherwise. The authors do not convince me that this is not the case in this setting.
-
The paper lacks motivation in existing literature and real-life applications. It is possible to come up with a number of simplifications to solve an otherwise too general optimization problem. However, these simplifications need to be grounded in some intuition, literature expertise from the same (or other) fields, or at least in empirical verifications and calculated metrics on true data. There seems to be little justification for the shared copula other than the possibility to do some nice, yet simple and straightforward math around it.
Questions for authors
-
What would be some real-life applications where we can expect to have a shared-copula or small violations from it (small as measured in Figure 2)?
-
How can the violation radius be estimated from data?
-
Why are we using the continuous improvement score in the utility function? Is there any literature to back it up to be significant? Do practitioners care about it, or does it pose some nice theoretical properties that were otherwise not mentioned in the paper?
Dear Reviewer e1d5,
Thank you so much for your followup. We want to express our deep appreciation for your constructive comments again, and will try our best to revise the paper according to your reviews. These suggestions are valuable to us for future submission! Wish you all the best!
Best regards,
Authors of paper 27621