- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper examines if a single importance ranking can be reused across multiple downstream decisions. The paper introduces a framework that distinguishes compatibility, consequence, auditability, and authorization. They present experiments comparing shared and decision-specific rankings and measure sharing costs and selective-reuse policies. The central finding is that decision-specific rankings can outperform shared rankings and that ranking reuse should be evaluated through its downstream consequences.
Critical strengths
- The paper considers a relevant practical question concerning the reuse of importance rankings across different decisions tasks.
- The authors conduct experiments using multiple datasets and provide detailed descriptions of their experimental setup.
Critical weaknesses
- The paper lacks significant technical novelty. The two theoretical propositions are simple observations. I do not see any substantive new theoretical or algorithmic contributions. Much of the ideas in the paper are established ideas from the field of decision-focused learning.
- The main experimental results are also unsurprising. They are the expected consequence of relaxing an optimization constraint. The experiments do not establish any interesting phenomenon or meaningful generalizable insight.
- The paper is poorly written. The work introduces unusual terminology and would benefit from leveraging standardized language from the field. The experiments use disparate settings with varying objectives and normalization procedures, making it difficult to identify what the results are meant to show and why they are interesting.
- Most of the paper is spent formalizing the framework via excessive notation and terminology. These largely correspond to existing concepts in preference aggregation, decision-focused learning, and statistical estimation, but this connection is omitted from the paper. It is unclear what novel methods the paper contributes.
Questions for authors
- Are there any new algorithmic or technical insights that can be drawn from the results?
- How do the terminology, models, and problem formulations in this paper relate to established formulations in the broader AI literature? What does the proposed framework contribute beyond these existing approaches?