- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Proposal-Certified Selection, a technique for selecting among multiple proposals in causal selection. The paper first shows that under noisy estimates of causal value, having more proposed candidates can hurt the expected regret under a simple plug-in estimator. Then, they propose a framework called PCS for selecting a candidate and validate it theoretically and empirically.
Critical strengths
- Interesting and important problem setting
I think the problem setting is interesting, and there could be value to the framework you are proposing. I encourage you to keep working on this topic and improve the paper and results.
Critical weaknesses
- Paper is very poorly-written
- Main theoretical results are very modest, some following trivially from assumptions or common knowledge
The paper in its current form is far from the bar for publication, in my view. This paper was very difficult to read and appears to be substantially written by AI. The abstract, introduction, and related work seemed completely AI-written with confusing logical structure, undefined terms, and convoluted sentence structure. The confusing writing appeared throughout the rest of the paper, to varying degrees. For instance, in 3.2, what does “The order may use generation randomness, retrieval, pretrained scores, or disjoint historical outcomes” mean? The order of what? The concept of is not precisely defined anywhere, and it’s not clear what you mean by “Let contain proposal randomness, fitted objects, the realized sequence, and pre-outcome design metadata.” The appendices largely read as if they were AI generated (even if not, they were very convoluted and difficult to read).
In Section 3, the “Notation” paragraph is duplicating most of the information in 3.1, except Section 3.1 more clearly explains what your notation means. I suggest removing this redundancy. (With the exception that 3.1 does not define the value estimates ).
Several of the main theorem statements are obvious or tautological. Thm. 1 is a very basic observation and not at the level of a theorem, in my view (it could also be significantly generalized beyond Gaussians, but I still think the key observation would not be interesting). The Theorem also does not specify what the “plug-in selector” is. Later theorems (e.g., Thm. 2) are tautological based on the definition of the selector, or follow directly from restrictive assumptions on the underlying random process (Thm 4). Overall, I did not find the theoretical results to be sufficiently novel or technically interesting to meet the bar for publication at this venue.
The figures are almost impossible to read. There were several parts of the figures that I could not read at all.
Questions for authors
Truthfully, I don't believe my main concerns can be resolved through Q&A or running more experiments. However, if you had to pick 1-2 theoretical results that are the "main result", which ones would they be, and what is their principal novelty?